-- Logs begin at Mon 2022-05-30 11:21:51 UTC, end at Mon 2022-05-30 12:30:36 UTC. -- May 30 11:49:04.253968 dv-cinder-840512-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 30 11:49:04.262630 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 30 11:49:04.266141 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 30 11:49:04 2022] *** May 30 11:49:04.266141 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 30 11:49:04.266141 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 30 11:49:04.266141 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: nodename: dv-cinder-840512-2-iscsi May 30 11:49:04.266141 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: machine: x86_64 May 30 11:49:04.266141 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: clock source: unix May 30 11:49:04.266141 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: pcre jit disabled May 30 11:49:04.266141 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: detected number of CPU cores: 4 May 30 11:49:04.266606 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: current working directory: / May 30 11:49:04.266606 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: detected binary path: /usr/bin/uwsgi-core May 30 11:49:04.266606 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: your processes number limit is 47680 May 30 11:49:04.266606 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: your memory page size is 4096 bytes May 30 11:49:04.266606 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: detected max file descriptor number: 2048 May 30 11:49:04.266606 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: lock engine: pthread robust mutexes May 30 11:49:04.266606 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: thunder lock: enabled May 30 11:49:04.267212 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 30 11:49:04.267212 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 30 11:49:04.312910 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: Python main interpreter initialized at 0x55e393d199c0 May 30 11:49:04.312910 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: python threads support enabled May 30 11:49:04.312910 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: your server socket listen backlog is limited to 100 connections May 30 11:49:04.312910 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: your mercy for graceful operations on workers is 90 seconds May 30 11:49:04.313403 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: mapped 403077 bytes (393 KB) for 2 cores May 30 11:49:04.313403 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: *** Operational MODE: preforking *** May 30 11:49:04.313643 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: *** uWSGI is running in multiple interpreter mode *** May 30 11:49:04.313643 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: spawned uWSGI master process (pid: 80154) May 30 11:49:04.313837 dv-cinder-840512-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 30 11:49:04.315540 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: spawned uWSGI worker 1 (pid: 80165, cores: 1) May 30 11:49:04.315540 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: spawned uWSGI worker 2 (pid: 80166, cores: 1) May 30 11:49:04.315540 dv-cinder-840512-2-iscsi devstack@keystone.service[80154]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 30 11:49:10.146072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 30 11:49:10.146072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: from cryptography.utils import int_from_bytes May 30 11:49:10.146860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 30 11:49:10.146860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: from cryptography.utils import int_from_bytes May 30 11:49:10.150213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 30 11:49:10.150213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: from cryptography.utils import int_from_bytes May 30 11:49:10.150664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 30 11:49:10.150664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: from cryptography.utils import int_from_bytes May 30 11:49:10.197736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 30 11:49:10.197926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 30 11:49:10.198105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] command line args: [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 30 11:49:10.198371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 30 11:49:10.198582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ================================================================================ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 30 11:49:10.198975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 30 11:49:10.199072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] admin_token = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.199179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 30 11:49:10.199382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] command line args: [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 30 11:49:10.199466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_dir = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.199654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 30 11:49:10.199780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.199860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ================================================================================ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 30 11:49:10.200079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] admin_token = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.200192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_source = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.200353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_dir = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.200469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] debug = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.200553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.200715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_source = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.200822 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] debug = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201869 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.201982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_config_append = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_config_append = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_dir = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_dir = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202799 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.202897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.203032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.203221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.203397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.203580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.203750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.203921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.204103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.204289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.204474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.204656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.204834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.205004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.205175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.205352 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.205527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.205714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.205857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.206033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.206208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.206376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.206554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.206739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.206909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.207078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.207254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.207429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.207583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.207734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.207849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.208000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.208116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.32/identity {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.208301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.32/identity {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.208428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] publish_errors = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.208542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] publish_errors = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.208662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.208776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.208894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.209032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.209147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.209259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.209380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.209558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.209682 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.209802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.209916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] standard_threads = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.210038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] standard_threads = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.210159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.210279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.210424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.210537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.210656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.210779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.210894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_journal = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_journal = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_json = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_json = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_stderr = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_stderr = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_syslog = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_syslog = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.211938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 30 11:49:10.212058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.212207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.212325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.212450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.212571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.212685 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.212806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.212920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.213093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.213209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.213321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.213444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.213636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.213759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.213873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.213985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.214629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.214815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.214944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.215066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.215181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.215294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.215412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.215534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.215658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.215780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.215894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.216999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.217121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.217235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.217391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.217510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.217623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.217743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.217743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.217999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.218123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.218246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.218358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.218470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.218594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.218711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.218824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.218936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.219048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.219161 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.219282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.219402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.219516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.219635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.219749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.219862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.220046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.220181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.220297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.220409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.220532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.220653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.220781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.220897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.221970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.222096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.222222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.222343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.222463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.222582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.222696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.222817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.222932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.223960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.224998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.225958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.226938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.227828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.228972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.229088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.229214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.229330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.229443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.229563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.229677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.230336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.230543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.230662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.230777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.230891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.231958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.232073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.232267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.232267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.232488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.232611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.232611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.232834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.232948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.233062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.233176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.233298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.233425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.233555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.233672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.233785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.233899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234126 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.234966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.235991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.236105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.236105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.236347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.236464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.external = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.236578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.236692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.236805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.external = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.236805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.237029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.237172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.237301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.237426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.237543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.password = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.237659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.password = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.237773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.token = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.237888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.token = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238127 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.238948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.239063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.239178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.239301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.239418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.239533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.239654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.239771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.239886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240126 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.240968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.241082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.241197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.241311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.241434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.241551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.241665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.241781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.241896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.242959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.243076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.243190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.243305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.243420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.243543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.243659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.243774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.243889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.244011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.244127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.244340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.244468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.244585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.244700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.244815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.244932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.245056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.245173 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.245288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.245438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.245555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.245670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.245793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.245911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.246035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.246150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.246265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.246380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.246507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.246624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.246739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.246903 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.247980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.248098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.248226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.248341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.248456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.248583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.248700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.248814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.248929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.249985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.250102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.250224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.250340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.250455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.250569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.250691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.250807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.250922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.251968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.252091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.252374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.252494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.252609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.252723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.252838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.252965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.253083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.253198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.253313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.253428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.253542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.253665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.253782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.253897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.254953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.255081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.255198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.255312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.255427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.255541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.255655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.255779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.255895 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.256959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.257073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.257196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.257313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.257427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.257543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.257658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.257772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.257895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.258823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.259958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.260075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.260203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.260320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.260435 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.260562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.260688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.260803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.260917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.261972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.262087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.262202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.262315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.262429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.262543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.262665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.262782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.262896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.263921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.264995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.265110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.265224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.265338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.265461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.265577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.265690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.265804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.265919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.266034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.266156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.266272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.266386 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.266500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.266614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.266728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.267176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.267298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.267414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.267528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.267643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.267758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.267882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.268000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.268115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.268313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.268430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.268545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.268673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.268790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.268905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.269951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.270075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.270191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.270306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.270420 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.270534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.270648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.270776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.270893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.271936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.272050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.272194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.272313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.272427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.272550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.272664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.272791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.272908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273252 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.273922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.274973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.275087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.275201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.275322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.275439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.275561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.275678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.275792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.275906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.276010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.276123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.276316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.276436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.276550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.276550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.277318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.277433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.277558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.277675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.277790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.277905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.278966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.279994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.280107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.280252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.280370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.280485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.280600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.280714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.280829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.280952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.281991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.282105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.282220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.282343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.282459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.282573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.282688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.282810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.282939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.283065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.283182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.283310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.283438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.283555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.283669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.283784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.283898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.284012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.284134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.284343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.284459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.284574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.284693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.284810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.284938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.285985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.286100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.286214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.286402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.286521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.286636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.286751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.286867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.286981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.287105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.287221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.287336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.287451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 30 11:49:10.287565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 30 11:49:10.287565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 30 11:49:10.315768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.315998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.316262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.321148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.321381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.321614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.325325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.331211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.358711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.358943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.360331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.360569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.414818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.414978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.464718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.464945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.465197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.465436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.533410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.533635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.533787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.533900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.534022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.534108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.534245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.534330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.534465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.534551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.534684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.534896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.558650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.558882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559632 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.559955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.560038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.560281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.560472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.574164 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.574351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.584703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.584939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.585989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.586072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.586198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.586410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 30 11:49:10.603770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.603981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 30 11:49:10.604236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.604364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.604449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 30 11:49:10.604544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 30 11:49:10.604798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.604944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.605072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 30 11:49:10.605165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.605377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.605550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.605634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.605866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.606126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.606382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.616711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.616988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.617258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.617382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.617543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.617654 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.617925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.618206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.626015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.626705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.629342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.629626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.629908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.630005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.630189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.630272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.630468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.630559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.630839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.630931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.631110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.631192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.631386 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.631468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.631653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.631734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.631922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.632001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.632218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.632402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.632546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.632737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.660457 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.661297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.662193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.662474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.662907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.663183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.667989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.668243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.668539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.668949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.669474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.669474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.675939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.676248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.676840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.677120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.680795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.681197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.681561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.681760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.682034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.682118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.682309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.682402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.682581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.682665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.682873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.682955 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.683341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.683854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.684112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.702041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.702317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.703563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.703835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.706957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.708373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.710598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.710887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.711970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.712283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.716041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.717599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.719642 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.721179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.724849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.725150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.725436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.725718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.725993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.726267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.726352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.726624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.726909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.727192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.727466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.727741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.746458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.746740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.747012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.747302 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.748063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.748462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.748730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.749006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.755353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.757271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.757429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.757429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.758612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.758885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.760435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.760717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.761039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.761474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.761768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.762284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.762564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.762842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.763131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.763427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.775429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.775709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.775991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.776297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.777089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.777404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.777682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.777860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.778152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.778437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.778718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.778998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.779269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.779546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.795937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.797893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.799217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.801167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.802556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.804186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.804475 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.805840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.806045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.806139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.806380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.807683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.808533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.809029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.814364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.815983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.816458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.818085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.819194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.819481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.819775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.820051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.821301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.821587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.821867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.822140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.829948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.830234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.830615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.832295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.832581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.832860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.838561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.838839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.839176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.839466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.839799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.840120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.840424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.840702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.840975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.841056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 30 11:49:10.841252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.841343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.841526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.841609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.841807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.841952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.842081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.842165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.842341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.842423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.842623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.842913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.843179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.843442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.843709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.843973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.844280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.844534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 30 11:49:10.880734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e393d199c0 pid: 80166 (default app) May 30 11:49:10.883388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e393d199c0 pid: 80165 (default app) May 30 11:49:10.884143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f571f-93d0-4487-85a5-d17d294e2e0b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:10.884461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f571f-93d0-4487-85a5-d17d294e2e0b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:10.884743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f571f-93d0-4487-85a5-d17d294e2e0b None None] PATH_INFO: `/v3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:10.889863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1/1] 10.222.0.32 () {50 vars in 791 bytes} [Mon May 30 11:49:10 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:18.992603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8ea0d-c217-48d9-a24c-c2eae922e9cc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:18.993140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8ea0d-c217-48d9-a24c-c2eae922e9cc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:18.993140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8ea0d-c217-48d9-a24c-c2eae922e9cc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:18.998091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1/2] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:49:19.003645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:19.004047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:19.004236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:19.072838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: INFO dbcounter [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] Registered counter for database keystone May 30 11:49:19.094954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG oslo_db.sqlalchemy.engines [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80166) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 30 11:49:19.101132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writer thread running {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 30 11:49:19.385256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG passlib.handlers.bcrypt [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80166) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 30 11:49:19.386180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG passlib.handlers.bcrypt [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80166) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 30 11:49:19.442056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:19.443715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:19.569058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 2/3] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 566 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 30 11:49:19.725524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8ea0d-c217-48d9-a24c-c2eae922e9cc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:19.796771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: INFO dbcounter [None req-11b8ea0d-c217-48d9-a24c-c2eae922e9cc None None] Registered counter for database keystone May 30 11:49:19.815931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG oslo_db.sqlalchemy.engines [None req-11b8ea0d-c217-48d9-a24c-c2eae922e9cc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80165) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 30 11:49:19.822159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writer thread running {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 30 11:49:20.047100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8ea0d-c217-48d9-a24c-c2eae922e9cc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:20.051524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae78dcc-7134-4310-af2f-f2c20327232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:20.052211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae78dcc-7134-4310-af2f-f2c20327232d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:20.052508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae78dcc-7134-4310-af2f-f2c20327232d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:20.052792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae78dcc-7134-4310-af2f-f2c20327232d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:20.061315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae78dcc-7134-4310-af2f-f2c20327232d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:20.113424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bae78dcc-7134-4310-af2f-f2c20327232d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 30 11:49:20.113824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae78dcc-7134-4310-af2f-f2c20327232d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:20.124247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 2/4] 10.222.0.32 () {58 vars in 1324 bytes} [Mon May 30 11:49:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 30 11:49:20.130281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:20.146017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f926c48-cce5-44d3-969c-6913a197e181 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:20.150427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962f39e-6357-44e8-970a-bee2b020d1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:20.152343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962f39e-6357-44e8-970a-bee2b020d1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:20.152829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962f39e-6357-44e8-970a-bee2b020d1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:20.153114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962f39e-6357-44e8-970a-bee2b020d1b4 admin admin] PATH_INFO: `/v3/projects/41eaf29f1f484186a95e357823daad72` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:20.154655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962f39e-6357-44e8-970a-bee2b020d1b4 admin admin] RBAC: Authorizing `identity:get_project(project_id=41eaf29f1f484186a95e357823daad72)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:20.206532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1962f39e-6357-44e8-970a-bee2b020d1b4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 30 11:49:20.206871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962f39e-6357-44e8-970a-bee2b020d1b4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:20.211492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 3/5] 10.222.0.32 () {56 vars in 1207 bytes} [Mon May 30 11:49:20 2022] GET /identity/v3/projects/41eaf29f1f484186a95e357823daad72 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:22.446787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6c6f8-49a0-4ca0-8ec0-116e2917e8c8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:22.447317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6c6f8-49a0-4ca0-8ec0-116e2917e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:22.447423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6c6f8-49a0-4ca0-8ec0-116e2917e8c8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:22.452117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 3/6] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:22.457659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e71b97-6928-4863-9bfc-aba2dcd4cb17 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:22.458076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e71b97-6928-4863-9bfc-aba2dcd4cb17 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:22.458234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e71b97-6928-4863-9bfc-aba2dcd4cb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:22.477774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c1e71b97-6928-4863-9bfc-aba2dcd4cb17 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:22.479400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c1e71b97-6928-4863-9bfc-aba2dcd4cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:22.489616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 4/7] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 30 11:49:22.514797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6c6f8-49a0-4ca0-8ec0-116e2917e8c8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:22.528329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6c6f8-49a0-4ca0-8ec0-116e2917e8c8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:22.532690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943ca40-ac0e-4485-b592-3213e81dd145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:22.533337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ca40-ac0e-4485-b592-3213e81dd145 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:22.533628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ca40-ac0e-4485-b592-3213e81dd145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:22.533909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ca40-ac0e-4485-b592-3213e81dd145 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:22.541910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943ca40-ac0e-4485-b592-3213e81dd145 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:22.543930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943ca40-ac0e-4485-b592-3213e81dd145 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:22.552939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 4/8] 10.222.0.32 () {58 vars in 1324 bytes} [Mon May 30 11:49:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 11:49:22.558773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e71b97-6928-4863-9bfc-aba2dcd4cb17 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:22.568057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e71b97-6928-4863-9bfc-aba2dcd4cb17 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:22.572767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb5dfc-6c76-4cc8-b5d3-301a3fc22bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:22.574268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb5dfc-6c76-4cc8-b5d3-301a3fc22bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:22.574554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb5dfc-6c76-4cc8-b5d3-301a3fc22bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:22.574842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb5dfc-6c76-4cc8-b5d3-301a3fc22bd2 admin admin] PATH_INFO: `/v3/users/a8ba729f356d47049a450132ac3851da` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:22.576306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb5dfc-6c76-4cc8-b5d3-301a3fc22bd2 admin admin] RBAC: Authorizing `identity:get_user(user_id=a8ba729f356d47049a450132ac3851da)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:22.578175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb5dfc-6c76-4cc8-b5d3-301a3fc22bd2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:22.579774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 5/9] 10.222.0.32 () {56 vars in 1198 bytes} [Mon May 30 11:49:22 2022] GET /identity/v3/users/a8ba729f356d47049a450132ac3851da => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:24.951897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58483683-fb41-473a-bd31-9430f1b61579 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:24.952429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58483683-fb41-473a-bd31-9430f1b61579 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:24.952530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58483683-fb41-473a-bd31-9430f1b61579 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:24.954200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 5/10] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:49:24.959506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8574-5e6e-457b-8b1d-a3999f3fbba6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:24.960010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8574-5e6e-457b-8b1d-a3999f3fbba6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:24.960255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8574-5e6e-457b-8b1d-a3999f3fbba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:24.973720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047d95-c575-4146-94d1-83d6fa1d3a23 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:24.974023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047d95-c575-4146-94d1-83d6fa1d3a23 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:24.974311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047d95-c575-4146-94d1-83d6fa1d3a23 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:24.975671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 6/11] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:49:24.981436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a4ca8574-5e6e-457b-8b1d-a3999f3fbba6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:24.981833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:24.982132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:24.982417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:24.982570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a4ca8574-5e6e-457b-8b1d-a3999f3fbba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:25.023984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG passlib.handlers.bcrypt [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80165) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 30 11:49:25.024448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG passlib.handlers.bcrypt [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80165) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 30 11:49:25.063889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 6/12] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:25.071679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:25.073439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:25.087119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca8574-5e6e-457b-8b1d-a3999f3fbba6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:25.112731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:25.175693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 7/13] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:49:25.192720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca8574-5e6e-457b-8b1d-a3999f3fbba6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:25.197742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:25.234232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713a00d-969e-4640-9d95-58a22895c90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:25.235515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a00d-969e-4640-9d95-58a22895c90c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:25.236015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a00d-969e-4640-9d95-58a22895c90c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:25.236562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a00d-969e-4640-9d95-58a22895c90c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:25.249962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713a00d-969e-4640-9d95-58a22895c90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a713a00d-969e-4640-9d95-58a22895c90c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:25.261315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:25.262476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:25.263309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 7/14] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:49:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:49:25.270226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713a00d-969e-4640-9d95-58a22895c90c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:25.301896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c230d-4f16-4276-acc2-05ca1a72ba03 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:25.304206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:25.343355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd6ac1-515b-4b8d-9554-adaa9f6b942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:25.344508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd6ac1-515b-4b8d-9554-adaa9f6b942a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:25.344803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd6ac1-515b-4b8d-9554-adaa9f6b942a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:25.345093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd6ac1-515b-4b8d-9554-adaa9f6b942a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:25.358295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:25.376865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713a00d-969e-4640-9d95-58a22895c90c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:25.413276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:25.413949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:25.414236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:25.414515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:25.415274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:25.415585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:25.417066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:25.418389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd6ac1-515b-4b8d-9554-adaa9f6b942a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:25.419920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd6ac1-515b-4b8d-9554-adaa9f6b942a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:25.429173 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 8/15] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:25.435178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:25.520679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 8/16] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 30 11:49:25.526755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd6ac1-515b-4b8d-9554-adaa9f6b942a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:25.529541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:25.537490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8008b6-bf63-49f4-9c72-230ad2e49a1e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:25.571224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0244d-d5a5-43ab-b6a4-a6ce98c07527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:25.572111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0244d-d5a5-43ab-b6a4-a6ce98c07527 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:25.572430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0244d-d5a5-43ab-b6a4-a6ce98c07527 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:25.572707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0244d-d5a5-43ab-b6a4-a6ce98c07527 None admin] PATH_INFO: `/v3/users/a8ba729f356d47049a450132ac3851da` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:25.574824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0244d-d5a5-43ab-b6a4-a6ce98c07527 None admin] RBAC: Authorizing `identity:get_user(user_id=a8ba729f356d47049a450132ac3851da)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:25.576350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0244d-d5a5-43ab-b6a4-a6ce98c07527 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:25.577999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 9/17] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:25 2022] GET /identity/v3/users/a8ba729f356d47049a450132ac3851da => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:25.583142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0244d-d5a5-43ab-b6a4-a6ce98c07527 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:25.599237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:25.629824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd6ac1-515b-4b8d-9554-adaa9f6b942a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:25.668356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:25.669164 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:25.669448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:25.669718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:25.688438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:25.690021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:25.692745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0244d-d5a5-43ab-b6a4-a6ce98c07527 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:49:25.711322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:25.712213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:25.713109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:49:25.713109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:49:25.713109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:49:25.713109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:25.713109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 9/18] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:49:25.719315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:25.731346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383d8c5-875a-48ea-bbfd-dcf45e2def9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:25.732229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d8c5-875a-48ea-bbfd-dcf45e2def9e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:25.732508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d8c5-875a-48ea-bbfd-dcf45e2def9e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:25.732794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383d8c5-875a-48ea-bbfd-dcf45e2def9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:25.733992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383d8c5-875a-48ea-bbfd-dcf45e2def9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:25.735470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383d8c5-875a-48ea-bbfd-dcf45e2def9e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:25.737148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 10/19] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:25.742682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383d8c5-875a-48ea-bbfd-dcf45e2def9e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:25.777923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:25.822023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842132a-e247-4982-8a00-d8f6a93fee08 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:25.828430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:25.849761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383d8c5-875a-48ea-bbfd-dcf45e2def9e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:25.857325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d71320-c301-4cde-8858-8e95583a4165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:25.857999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d71320-c301-4cde-8858-8e95583a4165 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:25.858290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d71320-c301-4cde-8858-8e95583a4165 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:25.858570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d71320-c301-4cde-8858-8e95583a4165 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:25.859297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d71320-c301-4cde-8858-8e95583a4165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:25.859609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d71320-c301-4cde-8858-8e95583a4165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:25.861038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d71320-c301-4cde-8858-8e95583a4165 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:25.879139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 10/20] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:25.883970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60db258-3634-470c-8ce4-ddbb432b92af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:25.884691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60db258-3634-470c-8ce4-ddbb432b92af None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:25.884979 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60db258-3634-470c-8ce4-ddbb432b92af None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:25.885256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60db258-3634-470c-8ce4-ddbb432b92af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:25.886334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60db258-3634-470c-8ce4-ddbb432b92af None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.domain.id=default, user.id=a8ba729f356d47049a450132ac3851da) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:25.886932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60db258-3634-470c-8ce4-ddbb432b92af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:25.888483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60db258-3634-470c-8ce4-ddbb432b92af None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:25.896879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 11/21] 10.222.0.32 () {56 vars in 1322 bytes} [Mon May 30 11:49:25 2022] GET /identity/v3/role_assignments?user.id=a8ba729f356d47049a450132ac3851da&scope.domain.id=default&role.id=4d21498572b14478a996ac538be5382e => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:28.691553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbae45-778a-41c2-a5e8-179000925174 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:28.692073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbae45-778a-41c2-a5e8-179000925174 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:28.692148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbae45-778a-41c2-a5e8-179000925174 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:28.693644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 11/22] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:28.705921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923fc3b-08a7-4656-abd5-5cb277fcc6bc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:28.706440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923fc3b-08a7-4656-abd5-5cb277fcc6bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:28.706514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923fc3b-08a7-4656-abd5-5cb277fcc6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:28.736067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8923fc3b-08a7-4656-abd5-5cb277fcc6bc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:28.737310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8923fc3b-08a7-4656-abd5-5cb277fcc6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:28.845928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 12/23] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:49:28.847741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66108043-fa37-49ba-aa97-ead66a16d2ac None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:28.848034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66108043-fa37-49ba-aa97-ead66a16d2ac None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:28.848340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66108043-fa37-49ba-aa97-ead66a16d2ac None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:28.849972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 12/24] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:28.856352 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5a632-8f44-4bfd-8539-eea2954d4737 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:28.856658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5a632-8f44-4bfd-8539-eea2954d4737 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:28.856926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5a632-8f44-4bfd-8539-eea2954d4737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:28.884478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66108043-fa37-49ba-aa97-ead66a16d2ac None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:28.889340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6eb5a632-8f44-4bfd-8539-eea2954d4737 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:28.895420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6eb5a632-8f44-4bfd-8539-eea2954d4737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:29.056335 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66108043-fa37-49ba-aa97-ead66a16d2ac None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:29.075139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 13/25] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:29.081482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bdaa6-1d23-4922-84c0-4c51c4db52c4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.081780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bdaa6-1d23-4922-84c0-4c51c4db52c4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.082069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bdaa6-1d23-4922-84c0-4c51c4db52c4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.083375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 14/26] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:29.089259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0ae0-ed56-4269-acf5-c9b172d7b009 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.089598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0ae0-ed56-4269-acf5-c9b172d7b009 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.089816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb0ae0-ed56-4269-acf5-c9b172d7b009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.101754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:29.116682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:29.146677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-24fb0ae0-ed56-4269-acf5-c9b172d7b009 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:29.147891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-24fb0ae0-ed56-4269-acf5-c9b172d7b009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:29.160225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8314cb4-bffb-4c15-bd3f-d4e5d974db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:29.161103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8314cb4-bffb-4c15-bd3f-d4e5d974db33 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.161393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8314cb4-bffb-4c15-bd3f-d4e5d974db33 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.161675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8314cb4-bffb-4c15-bd3f-d4e5d974db33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.187346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8314cb4-bffb-4c15-bd3f-d4e5d974db33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d8314cb4-bffb-4c15-bd3f-d4e5d974db33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:29.199904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:29.201949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:29.203816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:29.208402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 13/27] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:29.209170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8314cb4-bffb-4c15-bd3f-d4e5d974db33 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:29.308128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 15/28] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:29.309037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb0ae0-ed56-4269-acf5-c9b172d7b009 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:29.363205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8314cb4-bffb-4c15-bd3f-d4e5d974db33 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:29.402931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91329dc2-47ea-4870-9e13-a7b8e1aa4fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:29.403823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91329dc2-47ea-4870-9e13-a7b8e1aa4fd7 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.404046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91329dc2-47ea-4870-9e13-a7b8e1aa4fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.404397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91329dc2-47ea-4870-9e13-a7b8e1aa4fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.405166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91329dc2-47ea-4870-9e13-a7b8e1aa4fd7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:29.406672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91329dc2-47ea-4870-9e13-a7b8e1aa4fd7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:29.431340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb0ae0-ed56-4269-acf5-c9b172d7b009 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:29.467651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92115f4-12a7-4c1e-80d5-c4c8b4588aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:29.468712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92115f4-12a7-4c1e-80d5-c4c8b4588aca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.469218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92115f4-12a7-4c1e-80d5-c4c8b4588aca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.469781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92115f4-12a7-4c1e-80d5-c4c8b4588aca None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.470730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92115f4-12a7-4c1e-80d5-c4c8b4588aca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:29.471249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92115f4-12a7-4c1e-80d5-c4c8b4588aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:29.472856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92115f4-12a7-4c1e-80d5-c4c8b4588aca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:29.474497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:29.483526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 16/29] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:29.485376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecead8-dcb0-4158-aa95-ceed704bbf01 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.485867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecead8-dcb0-4158-aa95-ceed704bbf01 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.486341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecead8-dcb0-4158-aa95-ceed704bbf01 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.487790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 17/30] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:29.488706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecead8-dcb0-4158-aa95-ceed704bbf01 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:29.548343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 14/31] 10.222.0.32 () {60 vars in 1131 bytes} [Mon May 30 11:49:29 2022] POST /identity/v3/roles => generated 225 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:49:29.549782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91329dc2-47ea-4870-9e13-a7b8e1aa4fd7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:29.593233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:29.612867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecead8-dcb0-4158-aa95-ceed704bbf01 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:29.650600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68249566-4089-49a5-a47a-ee4e78f01ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:29.652193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68249566-4089-49a5-a47a-ee4e78f01ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.652530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68249566-4089-49a5-a47a-ee4e78f01ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.652892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68249566-4089-49a5-a47a-ee4e78f01ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.664668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68249566-4089-49a5-a47a-ee4e78f01ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:29.666293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68249566-4089-49a5-a47a-ee4e78f01ef1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:29.679503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91329dc2-47ea-4870-9e13-a7b8e1aa4fd7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-68249566-4089-49a5-a47a-ee4e78f01ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:49:29.687050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:29.687889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:29.688713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:49:29.688713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:49:29.688713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:49:29.688713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:29.688713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 18/32] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:29.694053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa534c75-55fb-42b7-a315-38204c823c2f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.694053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa534c75-55fb-42b7-a315-38204c823c2f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.694053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa534c75-55fb-42b7-a315-38204c823c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.708838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fa534c75-55fb-42b7-a315-38204c823c2f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:29.710144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fa534c75-55fb-42b7-a315-38204c823c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:29.734343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a745ce-733c-4b4f-a662-9b24ee577380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:29.736034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a745ce-733c-4b4f-a662-9b24ee577380 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.736531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a745ce-733c-4b4f-a662-9b24ee577380 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.737051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a745ce-733c-4b4f-a662-9b24ee577380 None admin] PATH_INFO: `/v3/users/a8ba729f356d47049a450132ac3851da` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.739562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a745ce-733c-4b4f-a662-9b24ee577380 None admin] RBAC: Authorizing `identity:get_user(user_id=a8ba729f356d47049a450132ac3851da)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:29.741826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a745ce-733c-4b4f-a662-9b24ee577380 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:29.744476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 15/33] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:29 2022] GET /identity/v3/users/a8ba729f356d47049a450132ac3851da => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:29.745907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a745ce-733c-4b4f-a662-9b24ee577380 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:29.761127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:29.788486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 19/34] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:29.789305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa534c75-55fb-42b7-a315-38204c823c2f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:29.874293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a745ce-733c-4b4f-a662-9b24ee577380 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:29.877649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:29.900783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa534c75-55fb-42b7-a315-38204c823c2f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:29.920289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:29.921307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.921770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.922234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.923150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:29.923873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:29.925559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:29.938889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232cfdb-4163-4446-a611-94dd92f68360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:29.939827 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232cfdb-4163-4446-a611-94dd92f68360 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:29.940205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232cfdb-4163-4446-a611-94dd92f68360 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:29.940565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232cfdb-4163-4446-a611-94dd92f68360 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:29.945161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232cfdb-4163-4446-a611-94dd92f68360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:29.945161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232cfdb-4163-4446-a611-94dd92f68360 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:29.945161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 20/36] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:29.945607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 16/36] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:29.945917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232cfdb-4163-4446-a611-94dd92f68360 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:29.949689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:30.038905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:30.102163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c07771-0712-489f-a49a-1e83670246b8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:30.121385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232cfdb-4163-4446-a611-94dd92f68360 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:30.148785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d11f3-8ced-47ef-9284-18fdc5a195f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:30.149844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d11f3-8ced-47ef-9284-18fdc5a195f3 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.151383 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d11f3-8ced-47ef-9284-18fdc5a195f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.152259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d11f3-8ced-47ef-9284-18fdc5a195f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.153462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d11f3-8ced-47ef-9284-18fdc5a195f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:30.155920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d11f3-8ced-47ef-9284-18fdc5a195f3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:30.166306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659fa89-a58f-4133-941d-58a5e6b32d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:30.167603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659fa89-a58f-4133-941d-58a5e6b32d13 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.167897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659fa89-a58f-4133-941d-58a5e6b32d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.168285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659fa89-a58f-4133-941d-58a5e6b32d13 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.169248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659fa89-a58f-4133-941d-58a5e6b32d13 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:30.170646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659fa89-a58f-4133-941d-58a5e6b32d13 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:30.193464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 17/37] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:49:29 2022] POST /identity/v3/projects => generated 301 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:49:30.194711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d11f3-8ced-47ef-9284-18fdc5a195f3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:30.222711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:30.316547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d11f3-8ced-47ef-9284-18fdc5a195f3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:30.386822 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0444-6503-4786-bd73-9ec9c0fb1312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:30.388180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0444-6503-4786-bd73-9ec9c0fb1312 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.388285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0444-6503-4786-bd73-9ec9c0fb1312 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.388566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0444-6503-4786-bd73-9ec9c0fb1312 None admin] PATH_INFO: `/v3/domains/default/users/a8ba729f356d47049a450132ac3851da/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.390791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0444-6503-4786-bd73-9ec9c0fb1312 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a8ba729f356d47049a450132ac3851da, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:30.392602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd0444-6503-4786-bd73-9ec9c0fb1312 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:30.402623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 21/38] 10.222.0.32 () {60 vars in 1131 bytes} [Mon May 30 11:49:29 2022] POST /identity/v3/roles => generated 219 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:49:30.403513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674d78a-e117-4ca2-a5d1-74acd364ede9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.403850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674d78a-e117-4ca2-a5d1-74acd364ede9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.404140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674d78a-e117-4ca2-a5d1-74acd364ede9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.405709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 22/39] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:30.411922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2704cb-366f-4eb6-8def-90dc939dd2bf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.412499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2704cb-366f-4eb6-8def-90dc939dd2bf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.413006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2704cb-366f-4eb6-8def-90dc939dd2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.419181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 18/40] 10.222.0.32 () {58 vars in 1361 bytes} [Mon May 30 11:49:30 2022] PUT /identity/v3/domains/default/users/a8ba729f356d47049a450132ac3851da/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:49:30.441518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9e2704cb-366f-4eb6-8def-90dc939dd2bf None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:30.443399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9e2704cb-366f-4eb6-8def-90dc939dd2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:30.484750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc017-820d-40d3-ab63-ef11c07eab84 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.485133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc017-820d-40d3-ab63-ef11c07eab84 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.485522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bc017-820d-40d3-ab63-ef11c07eab84 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.487233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 19/41] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:49:30.493465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0703f-b4b3-471f-b7b6-a919e91914d9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.493839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0703f-b4b3-471f-b7b6-a919e91914d9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.494201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0703f-b4b3-471f-b7b6-a919e91914d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.516671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9ee0703f-b4b3-471f-b7b6-a919e91914d9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:30.518974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9ee0703f-b4b3-471f-b7b6-a919e91914d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:30.542332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 23/42] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:30.572245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2704cb-366f-4eb6-8def-90dc939dd2bf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:30.621248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:30.645849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 20/43] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:30.674012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0703f-b4b3-471f-b7b6-a919e91914d9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:30.689889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:30.723258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2704cb-366f-4eb6-8def-90dc939dd2bf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:30.763146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dab7b9-52d4-434f-b31c-4429a5954060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:30.763832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dab7b9-52d4-434f-b31c-4429a5954060 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.764122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dab7b9-52d4-434f-b31c-4429a5954060 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.764447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dab7b9-52d4-434f-b31c-4429a5954060 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.765866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dab7b9-52d4-434f-b31c-4429a5954060 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:30.766732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dab7b9-52d4-434f-b31c-4429a5954060 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:30.782180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 24/44] 10.222.0.32 () {60 vars in 1131 bytes} [Mon May 30 11:49:30 2022] POST /identity/v3/roles => generated 223 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:49:30.799886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0703f-b4b3-471f-b7b6-a919e91914d9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:30.840978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46f9ec-b003-4cb6-bd85-54ff771778d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:30.841826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f9ec-b003-4cb6-bd85-54ff771778d7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:30.842122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f9ec-b003-4cb6-bd85-54ff771778d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:30.842407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f9ec-b003-4cb6-bd85-54ff771778d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:30.843746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46f9ec-b003-4cb6-bd85-54ff771778d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:30.845320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46f9ec-b003-4cb6-bd85-54ff771778d7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:30.847307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 21/45] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:30.853412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dab7b9-52d4-434f-b31c-4429a5954060 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:30.966689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:30.975358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dab7b9-52d4-434f-b31c-4429a5954060 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:31.017547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360fc60-60ec-4baf-ba7e-f3a193f70e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:31.018237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fc60-60ec-4baf-ba7e-f3a193f70e63 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:31.018525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fc60-60ec-4baf-ba7e-f3a193f70e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:31.018835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fc60-60ec-4baf-ba7e-f3a193f70e63 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:31.019561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360fc60-60ec-4baf-ba7e-f3a193f70e63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:31.021145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360fc60-60ec-4baf-ba7e-f3a193f70e63 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:31.052841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 25/46] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:49:30 2022] POST /identity/v3/projects => generated 312 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:49:32.789736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0f886-112d-4041-bdb6-47cd8425ed02 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:32.790282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0f886-112d-4041-bdb6-47cd8425ed02 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:32.790282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0f886-112d-4041-bdb6-47cd8425ed02 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:32.792325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 22/47] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:32.798040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc55e8-2b65-4dc9-83d1-926bb4b5e793 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:32.798440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc55e8-2b65-4dc9-83d1-926bb4b5e793 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:32.798613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc55e8-2b65-4dc9-83d1-926bb4b5e793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:32.815814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2ccc55e8-2b65-4dc9-83d1-926bb4b5e793 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:32.817050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2ccc55e8-2b65-4dc9-83d1-926bb4b5e793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:32.885915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 26/48] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:32.909965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0f886-112d-4041-bdb6-47cd8425ed02 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:32.991192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:33.023082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0f886-112d-4041-bdb6-47cd8425ed02 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:33.060599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1db63-3184-480e-98f6-96f007514f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:33.061409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1db63-3184-480e-98f6-96f007514f1e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.061714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1db63-3184-480e-98f6-96f007514f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.061989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1db63-3184-480e-98f6-96f007514f1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.072544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1db63-3184-480e-98f6-96f007514f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2dc1db63-3184-480e-98f6-96f007514f1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:33.080750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:33.081788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:33.082862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 23/49] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:33.087703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc55e8-2b65-4dc9-83d1-926bb4b5e793 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:33.167166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:33.189056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc55e8-2b65-4dc9-83d1-926bb4b5e793 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:33.224408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e72232-7423-4c62-bf7d-f4e0d9370bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:33.225212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72232-7423-4c62-bf7d-f4e0d9370bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.225570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72232-7423-4c62-bf7d-f4e0d9370bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.225856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72232-7423-4c62-bf7d-f4e0d9370bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.226812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e72232-7423-4c62-bf7d-f4e0d9370bb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:33.230646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e72232-7423-4c62-bf7d-f4e0d9370bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:33.230646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e72232-7423-4c62-bf7d-f4e0d9370bb8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:33.239605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 27/50] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:33.245785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1db63-3184-480e-98f6-96f007514f1e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:33.303725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48228717-e70e-4469-9213-0612bdeef42f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.304129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48228717-e70e-4469-9213-0612bdeef42f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.304450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48228717-e70e-4469-9213-0612bdeef42f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.305844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 28/51] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:49:33.311680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e2ff5-44fe-45c0-a70f-6eee04f4eed7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.312089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e2ff5-44fe-45c0-a70f-6eee04f4eed7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.312410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e2ff5-44fe-45c0-a70f-6eee04f4eed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.329349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b66e2ff5-44fe-45c0-a70f-6eee04f4eed7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:33.330567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b66e2ff5-44fe-45c0-a70f-6eee04f4eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:33.350517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1db63-3184-480e-98f6-96f007514f1e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:33.385414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232d0c8-1157-4c3b-b365-b8ef58c6ea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:33.386305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232d0c8-1157-4c3b-b365-b8ef58c6ea7d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.386590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232d0c8-1157-4c3b-b365-b8ef58c6ea7d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.386876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232d0c8-1157-4c3b-b365-b8ef58c6ea7d None admin] PATH_INFO: `/v3/users/a8ba729f356d47049a450132ac3851da` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.388413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232d0c8-1157-4c3b-b365-b8ef58c6ea7d None admin] RBAC: Authorizing `identity:get_user(user_id=a8ba729f356d47049a450132ac3851da)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:33.389886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232d0c8-1157-4c3b-b365-b8ef58c6ea7d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:33.391525 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 24/52] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:33 2022] GET /identity/v3/users/a8ba729f356d47049a450132ac3851da => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:33.397145 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232d0c8-1157-4c3b-b365-b8ef58c6ea7d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:33.398009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 29/53] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:49:33.399786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:33.422298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e2ff5-44fe-45c0-a70f-6eee04f4eed7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:33.494146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:33.501497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232d0c8-1157-4c3b-b365-b8ef58c6ea7d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:33.531042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e2ff5-44fe-45c0-a70f-6eee04f4eed7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:33.538237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621def-aae4-4c22-bb50-a8b6d9fa6c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:33.539048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621def-aae4-4c22-bb50-a8b6d9fa6c0d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.539336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621def-aae4-4c22-bb50-a8b6d9fa6c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.539615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd621def-aae4-4c22-bb50-a8b6d9fa6c0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.540828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd621def-aae4-4c22-bb50-a8b6d9fa6c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:33.542973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd621def-aae4-4c22-bb50-a8b6d9fa6c0d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:33.544647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 25/54] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:33.550287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621def-aae4-4c22-bb50-a8b6d9fa6c0d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:33.570032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7ea31-b035-4c66-a0fb-68dd2b1a9937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:33.570842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ea31-b035-4c66-a0fb-68dd2b1a9937 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.571133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ea31-b035-4c66-a0fb-68dd2b1a9937 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.571415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ea31-b035-4c66-a0fb-68dd2b1a9937 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.572662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7ea31-b035-4c66-a0fb-68dd2b1a9937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:33.574356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7ea31-b035-4c66-a0fb-68dd2b1a9937 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:33.576018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 30/55] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:33.581348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7ea31-b035-4c66-a0fb-68dd2b1a9937 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:33.621684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:33.654619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd621def-aae4-4c22-bb50-a8b6d9fa6c0d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:33.683135 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7ea31-b035-4c66-a0fb-68dd2b1a9937 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:33.687844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739256b8-fdb2-45ed-b33f-e5b4747b0239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:33.688550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739256b8-fdb2-45ed-b33f-e5b4747b0239 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.688844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739256b8-fdb2-45ed-b33f-e5b4747b0239 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.689130 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739256b8-fdb2-45ed-b33f-e5b4747b0239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.689978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739256b8-fdb2-45ed-b33f-e5b4747b0239 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.domain.id=default, user.id=a8ba729f356d47049a450132ac3851da) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:33.690305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739256b8-fdb2-45ed-b33f-e5b4747b0239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:33.691763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739256b8-fdb2-45ed-b33f-e5b4747b0239 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:33.700835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 26/56] 10.222.0.32 () {56 vars in 1322 bytes} [Mon May 30 11:49:33 2022] GET /identity/v3/role_assignments?user.id=a8ba729f356d47049a450132ac3851da&scope.domain.id=default&role.id=4d21498572b14478a996ac538be5382e => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:33.719016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd730233-4003-47e1-98c7-6620b649f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:33.719713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd730233-4003-47e1-98c7-6620b649f8f5 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:33.720002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd730233-4003-47e1-98c7-6620b649f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:33.720319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd730233-4003-47e1-98c7-6620b649f8f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:33.721186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd730233-4003-47e1-98c7-6620b649f8f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:33.722701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd730233-4003-47e1-98c7-6620b649f8f5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:33.732462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:33.752392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 31/57] 10.222.0.32 () {60 vars in 1140 bytes} [Mon May 30 11:49:33 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:49:36.055652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ede15e-b994-4625-9ded-a1b1d03b1ddd None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:36.056322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ede15e-b994-4625-9ded-a1b1d03b1ddd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:36.056322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ede15e-b994-4625-9ded-a1b1d03b1ddd None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:36.058036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 27/58] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:36.063274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b182a4-d1f9-4cd8-8863-eab6dc1f2150 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:36.063681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b182a4-d1f9-4cd8-8863-eab6dc1f2150 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:36.063850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b182a4-d1f9-4cd8-8863-eab6dc1f2150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:36.081008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b1b182a4-d1f9-4cd8-8863-eab6dc1f2150 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:36.082207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b1b182a4-d1f9-4cd8-8863-eab6dc1f2150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:36.149758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 32/59] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:36.174697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ede15e-b994-4625-9ded-a1b1d03b1ddd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:36.278412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ede15e-b994-4625-9ded-a1b1d03b1ddd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:36.315786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865cd44-05f1-4dc1-80b5-ff11993cff6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:36.316526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865cd44-05f1-4dc1-80b5-ff11993cff6a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:36.316813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865cd44-05f1-4dc1-80b5-ff11993cff6a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:36.317089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865cd44-05f1-4dc1-80b5-ff11993cff6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:36.318280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865cd44-05f1-4dc1-80b5-ff11993cff6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:36.319749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865cd44-05f1-4dc1-80b5-ff11993cff6a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:36.321407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 28/60] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:36.327615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b182a4-d1f9-4cd8-8863-eab6dc1f2150 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:36.427249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b182a4-d1f9-4cd8-8863-eab6dc1f2150 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:36.449073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:36.464955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026382a-d45d-4cd8-982d-4ff66beec414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:36.465946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026382a-d45d-4cd8-982d-4ff66beec414 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:36.466231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026382a-d45d-4cd8-982d-4ff66beec414 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:36.466508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026382a-d45d-4cd8-982d-4ff66beec414 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:36.467230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026382a-d45d-4cd8-982d-4ff66beec414 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:36.468734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026382a-d45d-4cd8-982d-4ff66beec414 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:36.493762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 33/61] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:49:36 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:49:39.712243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d59543-b4d5-40ac-a5a2-8cdc259386a8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:39.713449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d59543-b4d5-40ac-a5a2-8cdc259386a8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:39.714028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d59543-b4d5-40ac-a5a2-8cdc259386a8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:39.715914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 29/62] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:39.721844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c463-11e1-403c-9aa7-31ce4297ff1e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:39.722799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c463-11e1-403c-9aa7-31ce4297ff1e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:39.723285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c463-11e1-403c-9aa7-31ce4297ff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:39.744343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8603c463-11e1-403c-9aa7-31ce4297ff1e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:39.746091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8603c463-11e1-403c-9aa7-31ce4297ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:39.855211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 34/63] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:39.896811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d59543-b4d5-40ac-a5a2-8cdc259386a8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:39.900290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:40.044867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d59543-b4d5-40ac-a5a2-8cdc259386a8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:40.092026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6e3d2-dfd6-4d51-baec-b02d217ba44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:40.092889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6e3d2-dfd6-4d51-baec-b02d217ba44b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.093182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6e3d2-dfd6-4d51-baec-b02d217ba44b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.093471 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6e3d2-dfd6-4d51-baec-b02d217ba44b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.109664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6e3d2-dfd6-4d51-baec-b02d217ba44b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-73c6e3d2-dfd6-4d51-baec-b02d217ba44b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:40.123468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:40.124675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:40.125444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:40.129903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 30/64] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:40.137481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603c463-11e1-403c-9aa7-31ce4297ff1e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:40.208567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199800a0-4319-44e0-863d-2ae49390f29a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.208880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199800a0-4319-44e0-863d-2ae49390f29a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.209133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199800a0-4319-44e0-863d-2ae49390f29a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.210819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 31/65] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:49:40.228377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cd05e-12ee-469a-9980-dbeb79e5b629 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.228669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cd05e-12ee-469a-9980-dbeb79e5b629 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.229509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cd05e-12ee-469a-9980-dbeb79e5b629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.265186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7e6cd05e-12ee-469a-9980-dbeb79e5b629 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:40.266412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7e6cd05e-12ee-469a-9980-dbeb79e5b629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:40.289573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603c463-11e1-403c-9aa7-31ce4297ff1e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:40.321963 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:40.330265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:40.340819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:40.341806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.342312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.343051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 32/66] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:40.343257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.344236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:40.344742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:40.346532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:40.348240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5965a9-86e2-4471-aa07-c2650229bb28 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.348755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5965a9-86e2-4471-aa07-c2650229bb28 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.349060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5965a9-86e2-4471-aa07-c2650229bb28 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.350441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 33/67] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:40.361075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 35/68] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:40.363496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd633c5-7075-458e-b7e6-98522e693e6e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.363794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd633c5-7075-458e-b7e6-98522e693e6e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.364076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd633c5-7075-458e-b7e6-98522e693e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.373430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:40.400324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cfd633c5-7075-458e-b7e6-98522e693e6e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:40.402628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cfd633c5-7075-458e-b7e6-98522e693e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:40.484128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 34/69] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:40.485353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd633c5-7075-458e-b7e6-98522e693e6e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:40.505842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba328f9-5a9a-4694-8591-31e0109520cc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:40.551083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167f081-ea36-42d4-9a11-58339303a61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:40.551966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167f081-ea36-42d4-9a11-58339303a61b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.553231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167f081-ea36-42d4-9a11-58339303a61b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.553231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167f081-ea36-42d4-9a11-58339303a61b None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.575101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167f081-ea36-42d4-9a11-58339303a61b None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:40.575582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167f081-ea36-42d4-9a11-58339303a61b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:40.582839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 36/70] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:40 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:40.582839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b2f6-8263-45cd-8dc2-59a35b42b482 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.582839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b2f6-8263-45cd-8dc2-59a35b42b482 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.582839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307b2f6-8263-45cd-8dc2-59a35b42b482 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.583189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 37/71] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:40.584177 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307b2f6-8263-45cd-8dc2-59a35b42b482 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:40.612084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd633c5-7075-458e-b7e6-98522e693e6e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:40.618883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:40.639900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:40.654089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b3eb-eb00-4fa6-a2c4-49c6000882aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:40.654870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b3eb-eb00-4fa6-a2c4-49c6000882aa None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.655162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b3eb-eb00-4fa6-a2c4-49c6000882aa None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.655443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b3eb-eb00-4fa6-a2c4-49c6000882aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.667769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5b3eb-eb00-4fa6-a2c4-49c6000882aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f4f5b3eb-eb00-4fa6-a2c4-49c6000882aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:40.675498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:40.676395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:40.677154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 35/72] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:49:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:40.682334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b3eb-eb00-4fa6-a2c4-49c6000882aa None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:40.713770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307b2f6-8263-45cd-8dc2-59a35b42b482 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:40.752045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0381ef-f325-433b-8946-0281f3661b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:40.752977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0381ef-f325-433b-8946-0281f3661b4f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.753312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0381ef-f325-433b-8946-0281f3661b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.753595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0381ef-f325-433b-8946-0281f3661b4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.765432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0381ef-f325-433b-8946-0281f3661b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3f0381ef-f325-433b-8946-0281f3661b4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:40.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:40.774717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:40.777006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 38/73] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:49:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:40.777632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27870c-aacb-4803-90d6-0adcf8d33a06 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.777632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27870c-aacb-4803-90d6-0adcf8d33a06 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.777632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27870c-aacb-4803-90d6-0adcf8d33a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.797387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7f27870c-aacb-4803-90d6-0adcf8d33a06 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:40.798422 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b3eb-eb00-4fa6-a2c4-49c6000882aa None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:40.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7f27870c-aacb-4803-90d6-0adcf8d33a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:40.840688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6728a-eddf-45a1-8fdc-48a353d8e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:40.841416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6728a-eddf-45a1-8fdc-48a353d8e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:40.841710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6728a-eddf-45a1-8fdc-48a353d8e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:40.842107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6728a-eddf-45a1-8fdc-48a353d8e5c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:40.880736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 39/74] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:40.880736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27870c-aacb-4803-90d6-0adcf8d33a06 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:40.913533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:40.915799 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:40.936695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6728a-eddf-45a1-8fdc-48a353d8e5c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:40.938197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6728a-eddf-45a1-8fdc-48a353d8e5c5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:40.999924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27870c-aacb-4803-90d6-0adcf8d33a06 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.034777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.035458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.035751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.036027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.036808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:41.037113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.038701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.038800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 36/75] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:41.039469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6728a-eddf-45a1-8fdc-48a353d8e5c5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.047891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 40/76] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:41.048801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.127339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:41.127729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:41.146682 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6728a-eddf-45a1-8fdc-48a353d8e5c5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.151487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51e19e-7687-428a-9e8c-4c14b8be920e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.182641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.183305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.183585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.183860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.184393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de007b8-d237-4ae3-87ab-1b7f5aba0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.184645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:41.184959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.185192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de007b8-d237-4ae3-87ab-1b7f5aba0b93 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.185481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de007b8-d237-4ae3-87ab-1b7f5aba0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.185763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de007b8-d237-4ae3-87ab-1b7f5aba0b93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.186354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.186932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de007b8-d237-4ae3-87ab-1b7f5aba0b93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.188417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de007b8-d237-4ae3-87ab-1b7f5aba0b93 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.190199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 41/77] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:41.191579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2dd9-a2d3-45fc-8d6d-82eb59457e69 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.191875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2dd9-a2d3-45fc-8d6d-82eb59457e69 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.192198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2dd9-a2d3-45fc-8d6d-82eb59457e69 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.196223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 42/78] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:41.196223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2dd9-a2d3-45fc-8d6d-82eb59457e69 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.199646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 37/79] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:41.200528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.301798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f2018-8890-4165-a2f1-5484538d9027 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.303361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2dd9-a2d3-45fc-8d6d-82eb59457e69 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.334991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27a388-eec0-4ed0-afe7-f8d5073f60cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.335876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27a388-eec0-4ed0-afe7-f8d5073f60cc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.336181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27a388-eec0-4ed0-afe7-f8d5073f60cc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.336473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27a388-eec0-4ed0-afe7-f8d5073f60cc None admin] PATH_INFO: `/v3/users/a8ba729f356d47049a450132ac3851da` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.336565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37bddf-68fa-4e68-a7f0-bd7107f2b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.337403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37bddf-68fa-4e68-a7f0-bd7107f2b5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.337678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27a388-eec0-4ed0-afe7-f8d5073f60cc None admin] RBAC: Authorizing `identity:get_user(user_id=a8ba729f356d47049a450132ac3851da)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.337765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37bddf-68fa-4e68-a7f0-bd7107f2b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.337991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37bddf-68fa-4e68-a7f0-bd7107f2b5b5 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.339768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27a388-eec0-4ed0-afe7-f8d5073f60cc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.340198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37bddf-68fa-4e68-a7f0-bd7107f2b5b5 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.341457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 38/80] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/users/a8ba729f356d47049a450132ac3851da => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:41.341667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37bddf-68fa-4e68-a7f0-bd7107f2b5b5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.342208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27a388-eec0-4ed0-afe7-f8d5073f60cc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.348147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 43/81] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:41.348147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ff4c0-65f3-4a26-be2c-e35226e7489f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.348147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ff4c0-65f3-4a26-be2c-e35226e7489f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.348147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ff4c0-65f3-4a26-be2c-e35226e7489f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.359400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:41.361417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-445ff4c0-65f3-4a26-be2c-e35226e7489f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:41.362745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-445ff4c0-65f3-4a26-be2c-e35226e7489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:41.372456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:41.429993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 44/82] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:41.430915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ff4c0-65f3-4a26-be2c-e35226e7489f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.450059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27a388-eec0-4ed0-afe7-f8d5073f60cc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.484547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88095b-7f68-48c4-a74e-72f2c02dfca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.485224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88095b-7f68-48c4-a74e-72f2c02dfca9 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.485512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88095b-7f68-48c4-a74e-72f2c02dfca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.485803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88095b-7f68-48c4-a74e-72f2c02dfca9 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.486515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88095b-7f68-48c4-a74e-72f2c02dfca9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.487963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88095b-7f68-48c4-a74e-72f2c02dfca9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.516824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 39/83] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:49:41 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:49:41.518413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88095b-7f68-48c4-a74e-72f2c02dfca9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.540950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ff4c0-65f3-4a26-be2c-e35226e7489f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.595244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fda386-90c5-4318-9bd8-8cae13ea4d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.596281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fda386-90c5-4318-9bd8-8cae13ea4d28 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.596518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fda386-90c5-4318-9bd8-8cae13ea4d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.596854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fda386-90c5-4318-9bd8-8cae13ea4d28 None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.598262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fda386-90c5-4318-9bd8-8cae13ea4d28 None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.599813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fda386-90c5-4318-9bd8-8cae13ea4d28 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.601730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 45/84] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:41.602399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fda386-90c5-4318-9bd8-8cae13ea4d28 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.615756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:41.623459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:41.658175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88095b-7f68-48c4-a74e-72f2c02dfca9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.702921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.704305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.705272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.705732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.707034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] RBAC: Adding query filter params (role.id=932e429fa28248f285663a90fc880876, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7, user.id=156da27655884b45a9652caa634889b2) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:41.707755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.709846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.718007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 40/85] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/role_assignments?user.id=156da27655884b45a9652caa634889b2&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=932e429fa28248f285663a90fc880876 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:41.719080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:41.730431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fda386-90c5-4318-9bd8-8cae13ea4d28 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.781025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbfed5-a43a-467c-8911-ecd029914c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.781802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbfed5-a43a-467c-8911-ecd029914c8d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.782189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbfed5-a43a-467c-8911-ecd029914c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.782482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbfed5-a43a-467c-8911-ecd029914c8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.861658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83b1c8-e920-4035-b3ba-1fb6486149f7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:41.910099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:41.910283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbfed5-a43a-467c-8911-ecd029914c8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:41.910283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbfed5-a43a-467c-8911-ecd029914c8d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:41.916210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9139be-cdd5-4060-b085-1a4ed9d3d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:41.917076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9139be-cdd5-4060-b085-1a4ed9d3d838 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:41.917368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9139be-cdd5-4060-b085-1a4ed9d3d838 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:41.917647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9139be-cdd5-4060-b085-1a4ed9d3d838 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:41.928680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:41.935336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9139be-cdd5-4060-b085-1a4ed9d3d838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2b9139be-cdd5-4060-b085-1a4ed9d3d838 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:41.945110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:41.945989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:41.946821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:41.948675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 41/86] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:41.949759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9139be-cdd5-4060-b085-1a4ed9d3d838 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:42.056016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 46/87] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:42.057415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbfed5-a43a-467c-8911-ecd029914c8d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:42.094876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9139be-cdd5-4060-b085-1a4ed9d3d838 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:42.145798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6a99a-74d8-448f-8a9b-06b7185b5960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:42.146459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6a99a-74d8-448f-8a9b-06b7185b5960 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:42.146753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6a99a-74d8-448f-8a9b-06b7185b5960 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:42.147046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6a99a-74d8-448f-8a9b-06b7185b5960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:42.223499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbfed5-a43a-467c-8911-ecd029914c8d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:42.243248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:42.252454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:42.267799 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6a99a-74d8-448f-8a9b-06b7185b5960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:42.273981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:42.274238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6a99a-74d8-448f-8a9b-06b7185b5960 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:42.275020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:42.275476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:42.276687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:42.277800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:42.278278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:42.279693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:42.292775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 47/88] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:42.295157 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:42.470010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 42/89] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:42.471086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6a99a-74d8-448f-8a9b-06b7185b5960 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:42.508269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0e3c6-8960-4544-bbf9-06530da3cbd2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:42.593851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d5172-5ef5-416b-a6bf-0a13625ed526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:42.594734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d5172-5ef5-416b-a6bf-0a13625ed526 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:42.595030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d5172-5ef5-416b-a6bf-0a13625ed526 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:42.595315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d5172-5ef5-416b-a6bf-0a13625ed526 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:42.596566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d5172-5ef5-416b-a6bf-0a13625ed526 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:42.598166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d5172-5ef5-416b-a6bf-0a13625ed526 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:42.604087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 48/90] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:42 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:42.605362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:42.606229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d5172-5ef5-416b-a6bf-0a13625ed526 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:42.634190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6a99a-74d8-448f-8a9b-06b7185b5960 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:42.658700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:42.758260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d5172-5ef5-416b-a6bf-0a13625ed526 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:42.803372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf0b12-5d76-4588-87a8-bd89865b271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:42.805402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf0b12-5d76-4588-87a8-bd89865b271f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:42.806337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf0b12-5d76-4588-87a8-bd89865b271f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:42.807208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b490f1-e7f8-47cd-889b-5673665e8ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:42.808177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf0b12-5d76-4588-87a8-bd89865b271f None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:42.809806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b490f1-e7f8-47cd-889b-5673665e8ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:42.810665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf0b12-5d76-4588-87a8-bd89865b271f None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:42.811100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b490f1-e7f8-47cd-889b-5673665e8ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:42.812766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf0b12-5d76-4588-87a8-bd89865b271f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:42.813221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b490f1-e7f8-47cd-889b-5673665e8ec9 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:42.815571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 43/91] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:42 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:42.816498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b490f1-e7f8-47cd-889b-5673665e8ec9 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:42.818234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b490f1-e7f8-47cd-889b-5673665e8ec9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:42.820141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 49/92] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:42 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:42.821325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf0b12-5d76-4588-87a8-bd89865b271f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:42.833168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b490f1-e7f8-47cd-889b-5673665e8ec9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:42.972358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b490f1-e7f8-47cd-889b-5673665e8ec9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:42.984531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:43.022538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f467-8d14-4352-b4ab-521c060a2f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:43.022538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f467-8d14-4352-b4ab-521c060a2f16 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:43.022538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f467-8d14-4352-b4ab-521c060a2f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:43.022538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3f467-8d14-4352-b4ab-521c060a2f16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:43.203488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf0b12-5d76-4588-87a8-bd89865b271f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:43.286622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3f467-8d14-4352-b4ab-521c060a2f16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:43.288186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3f467-8d14-4352-b4ab-521c060a2f16 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:43.344908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885edc-6533-4e19-b467-41568375c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:43.345968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885edc-6533-4e19-b467-41568375c24e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:43.346463 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885edc-6533-4e19-b467-41568375c24e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:43.346965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885edc-6533-4e19-b467-41568375c24e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:43.348041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49885edc-6533-4e19-b467-41568375c24e None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7, user.id=a8ba729f356d47049a450132ac3851da) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:43.348587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49885edc-6533-4e19-b467-41568375c24e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:43.350695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49885edc-6533-4e19-b467-41568375c24e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:43.363536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 44/93] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:42 2022] GET /identity/v3/role_assignments?user.id=a8ba729f356d47049a450132ac3851da&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=4d21498572b14478a996ac538be5382e => generated 259 bytes in 543 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:43.364860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885edc-6533-4e19-b467-41568375c24e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:43.460587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:43.624197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 50/94] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 791 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:43.630872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f467-8d14-4352-b4ab-521c060a2f16 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:43.651939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:43.709409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885edc-6533-4e19-b467-41568375c24e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:43.780394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:43.781539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:43.781835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:43.782119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:43.782967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] RBAC: Adding query filter params (role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7, user.id=156da27655884b45a9652caa634889b2) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:43.783284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:43.784770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:43.798112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 45/95] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:43 2022] GET /identity/v3/role_assignments?user.id=156da27655884b45a9652caa634889b2&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 259 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:43.839383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3f467-8d14-4352-b4ab-521c060a2f16 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:43.898494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446e776-5b3d-4478-b752-8202fae61056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:43.899305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446e776-5b3d-4478-b752-8202fae61056 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:43.899595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446e776-5b3d-4478-b752-8202fae61056 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:43.899877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446e776-5b3d-4478-b752-8202fae61056 None admin] PATH_INFO: `/v3/projects/5e12fbee5aaf4b70a8f4b54dc8c9e8f2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:43.901758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446e776-5b3d-4478-b752-8202fae61056 None admin] RBAC: Authorizing `identity:get_project(project_id=5e12fbee5aaf4b70a8f4b54dc8c9e8f2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:43.903252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446e776-5b3d-4478-b752-8202fae61056 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:43.905269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 51/96] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:43 2022] GET /identity/v3/projects/5e12fbee5aaf4b70a8f4b54dc8c9e8f2 => generated 312 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:43.911079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:44.063641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:44.083414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d98fc-8bbb-4d8f-82c1-9bd443dbd75b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:44.136456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6035482-6407-4e19-82a9-7b0c9ce3d420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:44.137278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6035482-6407-4e19-82a9-7b0c9ce3d420 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:44.137569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6035482-6407-4e19-82a9-7b0c9ce3d420 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:44.137849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6035482-6407-4e19-82a9-7b0c9ce3d420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:44.138700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6035482-6407-4e19-82a9-7b0c9ce3d420 None admin] RBAC: Adding query filter params (role.id=932e429fa28248f285663a90fc880876, scope.project.id=5e12fbee5aaf4b70a8f4b54dc8c9e8f2, user.id=156da27655884b45a9652caa634889b2) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:44.139018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6035482-6407-4e19-82a9-7b0c9ce3d420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:44.140515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6035482-6407-4e19-82a9-7b0c9ce3d420 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:44.153249 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 46/97] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:43 2022] GET /identity/v3/role_assignments?user.id=156da27655884b45a9652caa634889b2&scope.project.id=5e12fbee5aaf4b70a8f4b54dc8c9e8f2&role.id=932e429fa28248f285663a90fc880876 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:44.306592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771c586-b3bb-44cd-9183-df73d8e92ce6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:44.307089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771c586-b3bb-44cd-9183-df73d8e92ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:44.307430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771c586-b3bb-44cd-9183-df73d8e92ce6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:44.309134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 52/98] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:44.326265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84a28-77f5-4581-8a4c-0e1fd9ebff8b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:44.326831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84a28-77f5-4581-8a4c-0e1fd9ebff8b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:44.326831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84a28-77f5-4581-8a4c-0e1fd9ebff8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:44.349465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e6d84a28-77f5-4581-8a4c-0e1fd9ebff8b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:44.350665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e6d84a28-77f5-4581-8a4c-0e1fd9ebff8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:44.446899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 47/99] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:44.489825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771c586-b3bb-44cd-9183-df73d8e92ce6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:44.624633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:44.644855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771c586-b3bb-44cd-9183-df73d8e92ce6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:44.668688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f74998-3224-407c-aa39-61641d3b2459 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:44.669521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f74998-3224-407c-aa39-61641d3b2459 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:44.669812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f74998-3224-407c-aa39-61641d3b2459 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:44.671663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 48/100] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:44.677512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d9e30-b65b-43a9-901a-fbb5abd8d037 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:44.677805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d9e30-b65b-43a9-901a-fbb5abd8d037 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:44.678090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d9e30-b65b-43a9-901a-fbb5abd8d037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:44.699436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbaf80-e148-4628-b147-4569d7d9b959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:44.700411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbaf80-e148-4628-b147-4569d7d9b959 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:44.700701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbaf80-e148-4628-b147-4569d7d9b959 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:44.700993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbaf80-e148-4628-b147-4569d7d9b959 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:44.708459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-898d9e30-b65b-43a9-901a-fbb5abd8d037 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:44.709700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-898d9e30-b65b-43a9-901a-fbb5abd8d037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:44.725221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbaf80-e148-4628-b147-4569d7d9b959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-22dbaf80-e148-4628-b147-4569d7d9b959 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:44.733878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:44.734774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:49:44.735572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:44.745327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 53/101] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:44.751458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbaf80-e148-4628-b147-4569d7d9b959 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:44.926298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbaf80-e148-4628-b147-4569d7d9b959 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:44.935841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 49/102] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:44.985549 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d9e30-b65b-43a9-901a-fbb5abd8d037 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:44.985832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:44.986464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:44.986754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:44.987038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:44.988054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:44.988399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:44.989803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:45.008186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 54/103] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:45.008186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:45.011300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:45.032555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:45.150980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937316-0693-4da0-9343-a006de5e3f1c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:45.186064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d9e30-b65b-43a9-901a-fbb5abd8d037 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:45.231848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00c8af-353f-45c1-95a1-926edc5cd7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:45.232782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00c8af-353f-45c1-95a1-926edc5cd7cd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:45.233068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00c8af-353f-45c1-95a1-926edc5cd7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:45.233347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00c8af-353f-45c1-95a1-926edc5cd7cd None admin] PATH_INFO: `/v3/users/dd7e8f5ac51f4e378717efe3e360bd9e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:45.253599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b171bd3-3f60-4315-bc67-f5c22ec0d450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:45.254327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b171bd3-3f60-4315-bc67-f5c22ec0d450 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:45.254610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b171bd3-3f60-4315-bc67-f5c22ec0d450 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:45.254887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b171bd3-3f60-4315-bc67-f5c22ec0d450 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:45.263485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00c8af-353f-45c1-95a1-926edc5cd7cd None admin] RBAC: Authorizing `identity:get_user(user_id=dd7e8f5ac51f4e378717efe3e360bd9e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:45.265059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00c8af-353f-45c1-95a1-926edc5cd7cd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:45.269816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 55/104] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:45 2022] GET /identity/v3/users/dd7e8f5ac51f4e378717efe3e360bd9e => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:45.270999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb07d6-fc61-473f-921d-7a286ced73b7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:45.271387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb07d6-fc61-473f-921d-7a286ced73b7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:45.271568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb07d6-fc61-473f-921d-7a286ced73b7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:45.278052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 56/105] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:45.278908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb07d6-fc61-473f-921d-7a286ced73b7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:45.282890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b171bd3-3f60-4315-bc67-f5c22ec0d450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9b171bd3-3f60-4315-bc67-f5c22ec0d450 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:45.293688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:45.294569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:45.295350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:45.302475 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 50/106] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:45.303804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371c1d-58a8-404c-893b-062ef6b3a0c7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:45.304092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371c1d-58a8-404c-893b-062ef6b3a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:45.304402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371c1d-58a8-404c-893b-062ef6b3a0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:45.341339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-97371c1d-58a8-404c-893b-062ef6b3a0c7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:45.342551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-97371c1d-58a8-404c-893b-062ef6b3a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:45.410084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:45.472698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb07d6-fc61-473f-921d-7a286ced73b7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:45.476457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 51/107] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:45.479152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97371c1d-58a8-404c-893b-062ef6b3a0c7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:45.482046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:45.524254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cec6f6-8f73-47aa-8e6f-04e23c8a3e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:45.524842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cec6f6-8f73-47aa-8e6f-04e23c8a3e4f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:45.525128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cec6f6-8f73-47aa-8e6f-04e23c8a3e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:45.525406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cec6f6-8f73-47aa-8e6f-04e23c8a3e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:45.609654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cec6f6-8f73-47aa-8e6f-04e23c8a3e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:45.611503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cec6f6-8f73-47aa-8e6f-04e23c8a3e4f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:45.673251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:45.695351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97371c1d-58a8-404c-893b-062ef6b3a0c7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:45.721218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 57/108] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:45.722418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cec6f6-8f73-47aa-8e6f-04e23c8a3e4f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:45.788881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:45.789581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:45.789873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:45.790154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:45.790934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:45.791770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:45.793220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:45.812612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 52/109] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:45.813492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:45.830800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cec6f6-8f73-47aa-8e6f-04e23c8a3e4f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:45.867829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932c221-4f63-457d-9da8-86019e793dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:45.869037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c221-4f63-457d-9da8-86019e793dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:45.869956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c221-4f63-457d-9da8-86019e793dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:45.870672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932c221-4f63-457d-9da8-86019e793dd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:45.872225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932c221-4f63-457d-9da8-86019e793dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:45.873906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932c221-4f63-457d-9da8-86019e793dd0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:45.875784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 58/110] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:45.876793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932c221-4f63-457d-9da8-86019e793dd0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:45.932540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:45.952695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:46.023162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dbbe4-002d-4d67-ad1f-e3accdcae6ea None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:46.067718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932c221-4f63-457d-9da8-86019e793dd0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:46.096848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379252a6-944f-4e1e-9fc1-4c70e3b45a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:46.097748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379252a6-944f-4e1e-9fc1-4c70e3b45a9d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.098072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379252a6-944f-4e1e-9fc1-4c70e3b45a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.098504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379252a6-944f-4e1e-9fc1-4c70e3b45a9d None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.100962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379252a6-944f-4e1e-9fc1-4c70e3b45a9d None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:46.102441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379252a6-944f-4e1e-9fc1-4c70e3b45a9d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:46.104636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 53/111] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:45 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:46.114333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379252a6-944f-4e1e-9fc1-4c70e3b45a9d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:46.153909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1267c-55d8-447b-90a6-3ccf4d8e145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:46.154817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1267c-55d8-447b-90a6-3ccf4d8e145f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.155116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1267c-55d8-447b-90a6-3ccf4d8e145f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.155404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1267c-55d8-447b-90a6-3ccf4d8e145f None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.156806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1267c-55d8-447b-90a6-3ccf4d8e145f None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:46.158296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1267c-55d8-447b-90a6-3ccf4d8e145f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:46.160288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 59/112] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:45 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:46.160919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1267c-55d8-447b-90a6-3ccf4d8e145f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:46.244765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379252a6-944f-4e1e-9fc1-4c70e3b45a9d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:46.307510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:46.322976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795da90-fcee-4ea0-b2c1-fa46f6fe0312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:46.323634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795da90-fcee-4ea0-b2c1-fa46f6fe0312 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.323925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795da90-fcee-4ea0-b2c1-fa46f6fe0312 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.324235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795da90-fcee-4ea0-b2c1-fa46f6fe0312 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.324979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795da90-fcee-4ea0-b2c1-fa46f6fe0312 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:46.326513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795da90-fcee-4ea0-b2c1-fa46f6fe0312 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:46.367472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:46.382725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 54/113] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:49:46 2022] POST /identity/v3/projects => generated 302 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:49:46.385404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1267c-55d8-447b-90a6-3ccf4d8e145f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:46.385625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795da90-fcee-4ea0-b2c1-fa46f6fe0312 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:46.430089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:46.431042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.432382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.432831 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.433885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] RBAC: Adding query filter params (role.id=c2921011b0a949eba8e918eb306cf45b, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7, user.id=dd7e8f5ac51f4e378717efe3e360bd9e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:46.434393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:46.436184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:46.448956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 60/114] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:46 2022] GET /identity/v3/role_assignments?user.id=dd7e8f5ac51f4e378717efe3e360bd9e&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=c2921011b0a949eba8e918eb306cf45b => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:46.586457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795da90-fcee-4ea0-b2c1-fa46f6fe0312 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:46.637326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862066e-1ffc-4119-854d-0291dc31afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:46.638240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862066e-1ffc-4119-854d-0291dc31afeb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.638529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862066e-1ffc-4119-854d-0291dc31afeb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.638812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862066e-1ffc-4119-854d-0291dc31afeb None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.645299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862066e-1ffc-4119-854d-0291dc31afeb None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:46.646854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862066e-1ffc-4119-854d-0291dc31afeb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:46.648761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 55/115] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:46 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:46.655585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:46.808552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17968716-fe39-4c9e-9b3d-be3bb3270fbf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:46.817367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054832d-a8e7-409e-a6ef-da4db61a1ae3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.820657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054832d-a8e7-409e-a6ef-da4db61a1ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.821133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054832d-a8e7-409e-a6ef-da4db61a1ae3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.822914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 56/116] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:49:46.831658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c2222-319b-48be-ad3e-9f465200503b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.832188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c2222-319b-48be-ad3e-9f465200503b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.832617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c2222-319b-48be-ad3e-9f465200503b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.865700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0b6c2222-319b-48be-ad3e-9f465200503b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:46.867172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b6c2222-319b-48be-ad3e-9f465200503b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:46.873564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120223d-7a00-4867-b031-74c43afab8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:46.874127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120223d-7a00-4867-b031-74c43afab8ee None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.874418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120223d-7a00-4867-b031-74c43afab8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.874705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120223d-7a00-4867-b031-74c43afab8ee None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/users/156da27655884b45a9652caa634889b2/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.880100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120223d-7a00-4867-b031-74c43afab8ee None admin] RBAC: Authorizing `identity:create_grant(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7, user_id=156da27655884b45a9652caa634889b2, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:46.881649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120223d-7a00-4867-b031-74c43afab8ee None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:46.907645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 61/117] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:49:46 2022] PUT /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/users/156da27655884b45a9652caa634889b2/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:49:46.945990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:46.967091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 57/118] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 30 11:49:46.983990 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6404d1-3d1e-417c-ad11-b733af7c818e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.984543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6404d1-3d1e-417c-ad11-b733af7c818e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.984625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6404d1-3d1e-417c-ad11-b733af7c818e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:46.985968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 62/119] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:46.991817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94c42e-0728-456b-9cfd-74e36be0b228 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:46.992109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94c42e-0728-456b-9cfd-74e36be0b228 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:46.992437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94c42e-0728-456b-9cfd-74e36be0b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:47.003687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6404d1-3d1e-417c-ad11-b733af7c818e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:47.017558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9d94c42e-0728-456b-9cfd-74e36be0b228 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:47.018773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9d94c42e-0728-456b-9cfd-74e36be0b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:47.063338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:47.132304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 58/120] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:47.174792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94c42e-0728-456b-9cfd-74e36be0b228 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:47.267679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6404d1-3d1e-417c-ad11-b733af7c818e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:47.309239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94c42e-0728-456b-9cfd-74e36be0b228 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:47.325551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:47.361364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3038d06-ec61-45c1-9ec3-54095fce8aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:47.362975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3038d06-ec61-45c1-9ec3-54095fce8aad None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:47.363824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3038d06-ec61-45c1-9ec3-54095fce8aad None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:47.365184 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b85b-fc2a-47e2-bac6-21e5780aebad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:47.365695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3038d06-ec61-45c1-9ec3-54095fce8aad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:47.366733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b85b-fc2a-47e2-bac6-21e5780aebad None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:47.367736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b85b-fc2a-47e2-bac6-21e5780aebad None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:47.368432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9b85b-fc2a-47e2-bac6-21e5780aebad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:47.391472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3038d06-ec61-45c1-9ec3-54095fce8aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e3038d06-ec61-45c1-9ec3-54095fce8aad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:47.399872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:47.400833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:47.401580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:47.403655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9b85b-fc2a-47e2-bac6-21e5780aebad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-53e9b85b-fc2a-47e2-bac6-21e5780aebad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:47.412492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:47.413355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:49:47.414103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:47.417079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 63/121] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:49:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:47.418557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 59/122] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:49:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:49:47.420152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed4985-2777-4057-aa65-55134e7f5d7a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:47.420731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed4985-2777-4057-aa65-55134e7f5d7a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:47.421222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed4985-2777-4057-aa65-55134e7f5d7a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:47.422912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b85b-fc2a-47e2-bac6-21e5780aebad None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:47.424473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 64/123] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:47.425412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed4985-2777-4057-aa65-55134e7f5d7a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:47.554489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:47.579202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9b85b-fc2a-47e2-bac6-21e5780aebad None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:47.622190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed4985-2777-4057-aa65-55134e7f5d7a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:47.622492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f84306-b425-4cf4-b891-9aa73818ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:47.623236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84306-b425-4cf4-b891-9aa73818ed10 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:47.623529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84306-b425-4cf4-b891-9aa73818ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:47.623896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84306-b425-4cf4-b891-9aa73818ed10 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:47.624702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f84306-b425-4cf4-b891-9aa73818ed10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:47.628189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f84306-b425-4cf4-b891-9aa73818ed10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:47.628189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f84306-b425-4cf4-b891-9aa73818ed10 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:47.646238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 60/124] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:47.647484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ec1b-064e-4615-aedc-4dfa1a534a7b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:47.647785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ec1b-064e-4615-aedc-4dfa1a534a7b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:47.648076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7ec1b-064e-4615-aedc-4dfa1a534a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:47.690970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-60c7ec1b-064e-4615-aedc-4dfa1a534a7b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:47.692228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-60c7ec1b-064e-4615-aedc-4dfa1a534a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:47.707998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:47.708727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:47.709019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:47.709307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:47.710076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:47.710395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:47.711800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:47.734630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 65/125] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:49:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:47.735530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:47.761333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:47.851885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ba09b-15b1-4d50-9b7a-ddbc0e97e6c5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:47.887920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 61/126] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:47.890950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7ec1b-064e-4615-aedc-4dfa1a534a7b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:47.895271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d3c1d-c744-46b5-a94c-60916331560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:47.896432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d3c1d-c744-46b5-a94c-60916331560c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:47.896877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d3c1d-c744-46b5-a94c-60916331560c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:47.898969 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d3c1d-c744-46b5-a94c-60916331560c None admin] PATH_INFO: `/v3/users/a8ba729f356d47049a450132ac3851da` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:47.901131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d3c1d-c744-46b5-a94c-60916331560c None admin] RBAC: Authorizing `identity:get_user(user_id=a8ba729f356d47049a450132ac3851da)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:47.907147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d3c1d-c744-46b5-a94c-60916331560c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:47.909047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 66/127] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:47 2022] GET /identity/v3/users/a8ba729f356d47049a450132ac3851da => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:47.925833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d3c1d-c744-46b5-a94c-60916331560c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:47.941347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:48.079004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7ec1b-064e-4615-aedc-4dfa1a534a7b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:48.080969 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d3c1d-c744-46b5-a94c-60916331560c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:48.146992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f85f09-d7ea-4033-b997-20759de3b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:48.148587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f85f09-d7ea-4033-b997-20759de3b83b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:48.149338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f85f09-d7ea-4033-b997-20759de3b83b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:48.151217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac6dbf-cc46-4cba-9d7c-41098d32c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:48.151690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f85f09-d7ea-4033-b997-20759de3b83b None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:48.153329 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6dbf-cc46-4cba-9d7c-41098d32c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:48.154132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f85f09-d7ea-4033-b997-20759de3b83b None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:48.154578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6dbf-cc46-4cba-9d7c-41098d32c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:48.156392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f85f09-d7ea-4033-b997-20759de3b83b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:48.156878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6dbf-cc46-4cba-9d7c-41098d32c8b1 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:48.158350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac6dbf-cc46-4cba-9d7c-41098d32c8b1 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:48.160256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 62/128] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:47 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:48.161989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac6dbf-cc46-4cba-9d7c-41098d32c8b1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:48.163902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 67/129] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:47 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:48.165018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f85f09-d7ea-4033-b997-20759de3b83b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:48.168309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac6dbf-cc46-4cba-9d7c-41098d32c8b1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:48.232276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:48.270279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:48.321042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac6dbf-cc46-4cba-9d7c-41098d32c8b1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:48.339764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f85f09-d7ea-4033-b997-20759de3b83b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:48.375995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106acb8d-33cb-4cea-8dd5-0fdcd7761df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:48.376879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106acb8d-33cb-4cea-8dd5-0fdcd7761df6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:48.377854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106acb8d-33cb-4cea-8dd5-0fdcd7761df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:48.378292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106acb8d-33cb-4cea-8dd5-0fdcd7761df6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:48.393712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106acb8d-33cb-4cea-8dd5-0fdcd7761df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-106acb8d-33cb-4cea-8dd5-0fdcd7761df6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:48.403125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:48.404653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:48.406480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:48.408656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 63/130] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:48.409767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106acb8d-33cb-4cea-8dd5-0fdcd7761df6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:48.424959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcf88c-147d-4f58-bb9f-562787ca93ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:48.425776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf88c-147d-4f58-bb9f-562787ca93ea None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:48.426070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf88c-147d-4f58-bb9f-562787ca93ea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:48.426355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf88c-147d-4f58-bb9f-562787ca93ea None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:48.432924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcf88c-147d-4f58-bb9f-562787ca93ea None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:48.434564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcf88c-147d-4f58-bb9f-562787ca93ea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:48.436348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 68/131] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:48 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:48.437134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcf88c-147d-4f58-bb9f-562787ca93ea None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:48.569167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcf88c-147d-4f58-bb9f-562787ca93ea None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:48.618196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:48.619067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:48.619427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:48.619722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:48.620524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:48.620961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:48.623540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:48.629968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:48.633419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 69/132] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:48.634336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:48.650201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:48.663219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106acb8d-33cb-4cea-8dd5-0fdcd7761df6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:48.734899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342743ff-8e4b-4bc8-a362-1cf06dd145fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:48.735872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342743ff-8e4b-4bc8-a362-1cf06dd145fa None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:48.736183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342743ff-8e4b-4bc8-a362-1cf06dd145fa None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:48.736529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342743ff-8e4b-4bc8-a362-1cf06dd145fa None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/users/a8ba729f356d47049a450132ac3851da/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:48.741661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342743ff-8e4b-4bc8-a362-1cf06dd145fa None admin] RBAC: Authorizing `identity:create_grant(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7, user_id=a8ba729f356d47049a450132ac3851da, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:48.745811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342743ff-8e4b-4bc8-a362-1cf06dd145fa None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:48.771981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 64/133] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:49:48 2022] PUT /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/users/a8ba729f356d47049a450132ac3851da/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:49:48.772866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342743ff-8e4b-4bc8-a362-1cf06dd145fa None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:48.777775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50898d45-4fba-480c-b10d-6adc47c6ffab None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:48.857627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e105a-5313-4e3f-9706-48dad8e3912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:48.858560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e105a-5313-4e3f-9706-48dad8e3912a None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:48.858854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e105a-5313-4e3f-9706-48dad8e3912a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:48.859136 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e105a-5313-4e3f-9706-48dad8e3912a None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/users/156da27655884b45a9652caa634889b2/roles/a511e28f566a49b4bacfbd0325aa44ed` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:48.865609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e105a-5313-4e3f-9706-48dad8e3912a None admin] RBAC: Authorizing `identity:create_grant(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7, user_id=156da27655884b45a9652caa634889b2, role_id=a511e28f566a49b4bacfbd0325aa44ed)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:48.867115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e105a-5313-4e3f-9706-48dad8e3912a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:48.898481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 70/134] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:49:48 2022] PUT /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/users/156da27655884b45a9652caa634889b2/roles/a511e28f566a49b4bacfbd0325aa44ed => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:49:48.929438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342743ff-8e4b-4bc8-a362-1cf06dd145fa None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:48.973303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80590389-a56f-4494-b1c2-1dbae79fc0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:48.974419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590389-a56f-4494-b1c2-1dbae79fc0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:48.974717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590389-a56f-4494-b1c2-1dbae79fc0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:48.975010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80590389-a56f-4494-b1c2-1dbae79fc0e7 None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:48.976691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80590389-a56f-4494-b1c2-1dbae79fc0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:48.978209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80590389-a56f-4494-b1c2-1dbae79fc0e7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:48.980906 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 65/135] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:48 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:48.986319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e105a-5313-4e3f-9706-48dad8e3912a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:49.100706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:49.162597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e105a-5313-4e3f-9706-48dad8e3912a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:49.211918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ef365-ccf0-4801-b5cf-cf9dae601993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:49.212764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ef365-ccf0-4801-b5cf-cf9dae601993 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:49.213575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ef365-ccf0-4801-b5cf-cf9dae601993 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:49.213861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ef365-ccf0-4801-b5cf-cf9dae601993 None admin] PATH_INFO: `/v3/projects/5e12fbee5aaf4b70a8f4b54dc8c9e8f2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:49.221211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ef365-ccf0-4801-b5cf-cf9dae601993 None admin] RBAC: Authorizing `identity:get_project(project_id=5e12fbee5aaf4b70a8f4b54dc8c9e8f2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:49.222730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ef365-ccf0-4801-b5cf-cf9dae601993 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:49.225082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 71/136] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:48 2022] GET /identity/v3/projects/5e12fbee5aaf4b70a8f4b54dc8c9e8f2 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:49.232027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80590389-a56f-4494-b1c2-1dbae79fc0e7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:49.263347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:49.368967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80590389-a56f-4494-b1c2-1dbae79fc0e7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:49.410996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b6e31-632f-492b-836e-86cdbf4791c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:49.411910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b6e31-632f-492b-836e-86cdbf4791c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:49.412223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b6e31-632f-492b-836e-86cdbf4791c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:49.412511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b6e31-632f-492b-836e-86cdbf4791c4 None admin] PATH_INFO: `/v3/projects/5e12fbee5aaf4b70a8f4b54dc8c9e8f2/users/156da27655884b45a9652caa634889b2/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:49.422587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0857a-b930-48ac-b967-5e4a3fb789a7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:49.422997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0857a-b930-48ac-b967-5e4a3fb789a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:49.423145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0857a-b930-48ac-b967-5e4a3fb789a7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:49.424740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 72/137] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:49.425684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b6e31-632f-492b-836e-86cdbf4791c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e12fbee5aaf4b70a8f4b54dc8c9e8f2, user_id=156da27655884b45a9652caa634889b2, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:49.428389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b6e31-632f-492b-836e-86cdbf4791c4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:49.431754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da63b6e-e3a5-45cc-8ae9-ce5ae3cdf310 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:49.432090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da63b6e-e3a5-45cc-8ae9-ce5ae3cdf310 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:49.432394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da63b6e-e3a5-45cc-8ae9-ce5ae3cdf310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:49.454288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9da63b6e-e3a5-45cc-8ae9-ce5ae3cdf310 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:49.458491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9da63b6e-e3a5-45cc-8ae9-ce5ae3cdf310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:49.459961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 66/138] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:49:49 2022] PUT /identity/v3/projects/5e12fbee5aaf4b70a8f4b54dc8c9e8f2/users/156da27655884b45a9652caa634889b2/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:49:49.544479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 73/139] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:49:49.567605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b6e31-632f-492b-836e-86cdbf4791c4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:49.585021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d51f39-af16-415c-9541-bdbb71847ab3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:49.585332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d51f39-af16-415c-9541-bdbb71847ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:49.585598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d51f39-af16-415c-9541-bdbb71847ab3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:49.587057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 74/140] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:49.599425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939545e2-b421-43d4-a9dc-98edde810377 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:49.599757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939545e2-b421-43d4-a9dc-98edde810377 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:49.599995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939545e2-b421-43d4-a9dc-98edde810377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:49.626471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-939545e2-b421-43d4-a9dc-98edde810377 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:49.629234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-939545e2-b421-43d4-a9dc-98edde810377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:49.743503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:49.765971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:49.771143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b6e31-632f-492b-836e-86cdbf4791c4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:49.787857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 75/141] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:49:49.818960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939545e2-b421-43d4-a9dc-98edde810377 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:49.842415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9b5c5-f4b2-413c-b4ee-2fc6bd04756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:49.843242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9b5c5-f4b2-413c-b4ee-2fc6bd04756a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:49.843531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9b5c5-f4b2-413c-b4ee-2fc6bd04756a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:49.843815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9b5c5-f4b2-413c-b4ee-2fc6bd04756a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:49.862619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9b5c5-f4b2-413c-b4ee-2fc6bd04756a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-51e9b5c5-f4b2-413c-b4ee-2fc6bd04756a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:49.871849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:49.872954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:49:49.873882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:49.875911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 67/142] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:49.882803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9b5c5-f4b2-413c-b4ee-2fc6bd04756a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:50.027102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939545e2-b421-43d4-a9dc-98edde810377 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:50.078008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1c79a-cc7d-4ddb-9af4-65ef93135656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:50.078861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1c79a-cc7d-4ddb-9af4-65ef93135656 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.079326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1c79a-cc7d-4ddb-9af4-65ef93135656 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.079766 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1c79a-cc7d-4ddb-9af4-65ef93135656 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.081347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1c79a-cc7d-4ddb-9af4-65ef93135656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:50.082985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1c79a-cc7d-4ddb-9af4-65ef93135656 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:50.089233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 76/143] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:50.090855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba6ab0-3f92-4d58-b662-fc543dad7bec None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.091363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba6ab0-3f92-4d58-b662-fc543dad7bec None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.091791 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba6ab0-3f92-4d58-b662-fc543dad7bec None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.093576 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 77/144] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:50.103458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba6ab0-3f92-4d58-b662-fc543dad7bec None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:50.109997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9b5c5-f4b2-413c-b4ee-2fc6bd04756a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:50.158258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da0442-4712-44e9-85dc-19b032c342df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:50.159203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da0442-4712-44e9-85dc-19b032c342df None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.159643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da0442-4712-44e9-85dc-19b032c342df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.159931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da0442-4712-44e9-85dc-19b032c342df None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.160712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da0442-4712-44e9-85dc-19b032c342df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:50.161185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da0442-4712-44e9-85dc-19b032c342df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:50.162733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da0442-4712-44e9-85dc-19b032c342df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:50.183878 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 68/145] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:49:50.185699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405bcbc-ed81-4e27-9518-4ad35323a528 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.186124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405bcbc-ed81-4e27-9518-4ad35323a528 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.186554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405bcbc-ed81-4e27-9518-4ad35323a528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.205988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:50.206175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:50.234951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0405bcbc-ed81-4e27-9518-4ad35323a528 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:50.236669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0405bcbc-ed81-4e27-9518-4ad35323a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:50.259118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba6ab0-3f92-4d58-b662-fc543dad7bec None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:50.331206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f549355-99e4-4eef-81c4-52add5d54afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:50.332291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549355-99e4-4eef-81c4-52add5d54afe None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.332775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549355-99e4-4eef-81c4-52add5d54afe None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.333309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549355-99e4-4eef-81c4-52add5d54afe None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.334241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f549355-99e4-4eef-81c4-52add5d54afe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:50.336816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f549355-99e4-4eef-81c4-52add5d54afe None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:50.376720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 78/146] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:49:50 2022] POST /identity/v3/users => generated 283 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:49:50.377875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f549355-99e4-4eef-81c4-52add5d54afe None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:50.383133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 69/147] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:50.470069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405bcbc-ed81-4e27-9518-4ad35323a528 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:50.563520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f549355-99e4-4eef-81c4-52add5d54afe None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:50.606247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405bcbc-ed81-4e27-9518-4ad35323a528 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:50.613390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:50.643255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:50.653093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72fb61-7d51-42bf-908c-4caa0fe4cd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:50.654389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72fb61-7d51-42bf-908c-4caa0fe4cd3a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.654843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72fb61-7d51-42bf-908c-4caa0fe4cd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.655067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04951f0a-6d16-4e25-95d5-3a3d56cb6e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:50.655481 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72fb61-7d51-42bf-908c-4caa0fe4cd3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.656132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04951f0a-6d16-4e25-95d5-3a3d56cb6e0a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.658203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04951f0a-6d16-4e25-95d5-3a3d56cb6e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.658203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04951f0a-6d16-4e25-95d5-3a3d56cb6e0a None admin] PATH_INFO: `/v3/users/dd7e8f5ac51f4e378717efe3e360bd9e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.658203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04951f0a-6d16-4e25-95d5-3a3d56cb6e0a None admin] RBAC: Authorizing `identity:get_user(user_id=dd7e8f5ac51f4e378717efe3e360bd9e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:50.659592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04951f0a-6d16-4e25-95d5-3a3d56cb6e0a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:50.664304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 79/148] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:50 2022] GET /identity/v3/users/dd7e8f5ac51f4e378717efe3e360bd9e => generated 289 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:50.670195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04951f0a-6d16-4e25-95d5-3a3d56cb6e0a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:50.681191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72fb61-7d51-42bf-908c-4caa0fe4cd3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-bd72fb61-7d51-42bf-908c-4caa0fe4cd3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:50.697024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:50.698610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:50.699413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 70/149] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:49:50.711567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72fb61-7d51-42bf-908c-4caa0fe4cd3a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:50.879934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04951f0a-6d16-4e25-95d5-3a3d56cb6e0a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:50.906173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72fb61-7d51-42bf-908c-4caa0fe4cd3a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:50.961854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b5867-4d65-4669-8413-c04743d0a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:50.962743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b5867-4d65-4669-8413-c04743d0a59b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.963059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b5867-4d65-4669-8413-c04743d0a59b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.963346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b5867-4d65-4669-8413-c04743d0a59b None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.965573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b5867-4d65-4669-8413-c04743d0a59b None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:50.968555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b5867-4d65-4669-8413-c04743d0a59b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:50.971071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:50.971756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:50.972046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:50.972355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:50.973132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:50.973451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:50.974843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:50.980758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 80/150] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:50 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:50.988918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 71/151] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:50.995196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b5867-4d65-4669-8413-c04743d0a59b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:51.006672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:51.108547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:51.121946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:51.155825 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609a5f7-fb1f-4a91-8b6d-732868877496 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:51.194464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714c81-f9f2-4ef3-8d79-aa241a3ed0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:51.195642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714c81-f9f2-4ef3-8d79-aa241a3ed0b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:51.196077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714c81-f9f2-4ef3-8d79-aa241a3ed0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:51.196526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50714c81-f9f2-4ef3-8d79-aa241a3ed0b0 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/users/dd7e8f5ac51f4e378717efe3e360bd9e/roles/c2921011b0a949eba8e918eb306cf45b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:51.198850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50714c81-f9f2-4ef3-8d79-aa241a3ed0b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7, user_id=dd7e8f5ac51f4e378717efe3e360bd9e, role_id=c2921011b0a949eba8e918eb306cf45b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:51.200527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50714c81-f9f2-4ef3-8d79-aa241a3ed0b0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:51.219485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 72/152] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:49:50 2022] PUT /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/users/dd7e8f5ac51f4e378717efe3e360bd9e/roles/c2921011b0a949eba8e918eb306cf45b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:49:51.252401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b5867-4d65-4669-8413-c04743d0a59b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:51.306708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce82d1d-4226-495d-a256-6ad19e3c43ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:51.307653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce82d1d-4226-495d-a256-6ad19e3c43ff None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:51.307946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce82d1d-4226-495d-a256-6ad19e3c43ff None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:51.308254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce82d1d-4226-495d-a256-6ad19e3c43ff None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:51.309583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce82d1d-4226-495d-a256-6ad19e3c43ff None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:51.311152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce82d1d-4226-495d-a256-6ad19e3c43ff None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:51.313252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 81/153] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:50 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:51.318977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714c81-f9f2-4ef3-8d79-aa241a3ed0b0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:51.453071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50714c81-f9f2-4ef3-8d79-aa241a3ed0b0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:51.484185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:51.496206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f80a6-1556-4dc5-b0e1-de04b49023af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:51.497644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f80a6-1556-4dc5-b0e1-de04b49023af None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:51.500584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f80a6-1556-4dc5-b0e1-de04b49023af None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:51.500902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f80a6-1556-4dc5-b0e1-de04b49023af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:51.586026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f80a6-1556-4dc5-b0e1-de04b49023af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:51.587543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f80a6-1556-4dc5-b0e1-de04b49023af None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:51.696919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 73/154] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:51.703704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce82d1d-4226-495d-a256-6ad19e3c43ff None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:51.818255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce82d1d-4226-495d-a256-6ad19e3c43ff None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:51.844139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:51.860919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050c41c-513d-4711-8657-7ce28ebcc153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:51.861813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c41c-513d-4711-8657-7ce28ebcc153 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:51.862096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c41c-513d-4711-8657-7ce28ebcc153 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:51.862373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c41c-513d-4711-8657-7ce28ebcc153 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:51.863637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050c41c-513d-4711-8657-7ce28ebcc153 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:51.865130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050c41c-513d-4711-8657-7ce28ebcc153 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:51.867042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 82/155] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:51 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:51.872908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f80a6-1556-4dc5-b0e1-de04b49023af None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:51.891741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:51.996685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f80a6-1556-4dc5-b0e1-de04b49023af None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:52.033636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5865a01-52a9-4001-a19f-e6f3f71b10aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:52.034583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5865a01-52a9-4001-a19f-e6f3f71b10aa None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:52.035005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5865a01-52a9-4001-a19f-e6f3f71b10aa None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:52.035292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5865a01-52a9-4001-a19f-e6f3f71b10aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:52.036149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5865a01-52a9-4001-a19f-e6f3f71b10aa None admin] RBAC: Adding query filter params (role.id=932e429fa28248f285663a90fc880876, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7, user.id=156da27655884b45a9652caa634889b2) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:52.036527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5865a01-52a9-4001-a19f-e6f3f71b10aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:52.037949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5865a01-52a9-4001-a19f-e6f3f71b10aa None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:52.044788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 74/156] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:51 2022] GET /identity/v3/role_assignments?user.id=156da27655884b45a9652caa634889b2&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=932e429fa28248f285663a90fc880876 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:52.754438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df79c26-66b5-4cd0-8351-70e2ccff08f1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:52.755111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df79c26-66b5-4cd0-8351-70e2ccff08f1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:52.755365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df79c26-66b5-4cd0-8351-70e2ccff08f1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:52.758083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 83/157] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:52.766469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd20d-5702-425e-9a14-4c2dedcadf7a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:52.768091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd20d-5702-425e-9a14-4c2dedcadf7a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:52.768846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbd20d-5702-425e-9a14-4c2dedcadf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:52.802566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7fcbd20d-5702-425e-9a14-4c2dedcadf7a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:52.804993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7fcbd20d-5702-425e-9a14-4c2dedcadf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:52.846010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80283e7a-98ef-4cfd-9b28-74bb21255ee6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:52.846865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80283e7a-98ef-4cfd-9b28-74bb21255ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:52.847577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80283e7a-98ef-4cfd-9b28-74bb21255ee6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:52.850271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 84/158] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:52.856845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6b537-1f54-468e-a285-2fae41e9b58e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:52.857735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6b537-1f54-468e-a285-2fae41e9b58e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:52.858521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6b537-1f54-468e-a285-2fae41e9b58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:52.897219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ccd6b537-1f54-468e-a285-2fae41e9b58e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:52.898434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ccd6b537-1f54-468e-a285-2fae41e9b58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:52.964833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 75/159] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:53.002360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85712103-b9e6-42de-a6d1-3b065a5f9eb7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:53.003311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85712103-b9e6-42de-a6d1-3b065a5f9eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:53.004064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85712103-b9e6-42de-a6d1-3b065a5f9eb7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:53.006796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 76/160] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:53.014351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee203cae-7405-4940-af72-29c7eab4d3c1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:53.015206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee203cae-7405-4940-af72-29c7eab4d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:53.015992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee203cae-7405-4940-af72-29c7eab4d3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:53.045915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 85/161] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:49:53.065857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ee203cae-7405-4940-af72-29c7eab4d3c1 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:53.066608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6b537-1f54-468e-a285-2fae41e9b58e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:53.068393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ee203cae-7405-4940-af72-29c7eab4d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:53.124599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:53.241876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 77/162] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:49:53.243131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee203cae-7405-4940-af72-29c7eab4d3c1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:53.316113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6b537-1f54-468e-a285-2fae41e9b58e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:53.357049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:53.392387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1592fd-655e-4e77-92f0-44ba4485ab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:53.393493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592fd-655e-4e77-92f0-44ba4485ab73 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:53.393926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592fd-655e-4e77-92f0-44ba4485ab73 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:53.394349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592fd-655e-4e77-92f0-44ba4485ab73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:53.419800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1592fd-655e-4e77-92f0-44ba4485ab73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fc1592fd-655e-4e77-92f0-44ba4485ab73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:53.443029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:53.444013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:49:53.444815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:53.448067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 86/163] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:49:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:49:53.452185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1592fd-655e-4e77-92f0-44ba4485ab73 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:53.482739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee203cae-7405-4940-af72-29c7eab4d3c1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:53.605499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105dd93-8ee4-47ad-847c-9c468660afbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:53.606633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105dd93-8ee4-47ad-847c-9c468660afbe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:53.607139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105dd93-8ee4-47ad-847c-9c468660afbe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:53.607616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105dd93-8ee4-47ad-847c-9c468660afbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:53.629755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:53.638195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1592fd-655e-4e77-92f0-44ba4485ab73 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:53.641154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105dd93-8ee4-47ad-847c-9c468660afbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6105dd93-8ee4-47ad-847c-9c468660afbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:53.657789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:53.658689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:53.659453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:53.661858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 78/164] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:53.662970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105dd93-8ee4-47ad-847c-9c468660afbe None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:53.735851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6f994-f571-44ed-9b0f-f433d41dc1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:53.736691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6f994-f571-44ed-9b0f-f433d41dc1be None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:53.737139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6f994-f571-44ed-9b0f-f433d41dc1be None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:53.737560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6f994-f571-44ed-9b0f-f433d41dc1be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:53.761404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6f994-f571-44ed-9b0f-f433d41dc1be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-33e6f994-f571-44ed-9b0f-f433d41dc1be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:53.770073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:53.771207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:53.771976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:53.781552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 87/165] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:49:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:53.782610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6f994-f571-44ed-9b0f-f433d41dc1be None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:53.880188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:53.910794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105dd93-8ee4-47ad-847c-9c468660afbe None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:53.991282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6f994-f571-44ed-9b0f-f433d41dc1be None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:54.003665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:54.004394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:54.004689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:54.004974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:54.005745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:54.006061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:54.007458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:54.030560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:54.031242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:54.031415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 79/166] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:49:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:54.031824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:54.032265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:54.032547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:54.033500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:54.033954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:54.035510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:54.045851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 88/167] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:54.047005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:54.207142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bca16-bbc4-4c3c-afc5-bc5d7d45aa06 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:54.208723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:54.214661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:54.272756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1de1b-c761-442e-ae3b-a5f5c6ba45d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:54.274025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1de1b-c761-442e-ae3b-a5f5c6ba45d2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:54.274554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1de1b-c761-442e-ae3b-a5f5c6ba45d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:54.275038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1de1b-c761-442e-ae3b-a5f5c6ba45d2 None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:54.280896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1de1b-c761-442e-ae3b-a5f5c6ba45d2 None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:54.282686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf1de1b-c761-442e-ae3b-a5f5c6ba45d2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:54.284697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 89/168] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:54 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:54.285694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1de1b-c761-442e-ae3b-a5f5c6ba45d2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:54.342376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddcedf-c43f-4301-857c-09115238bfe9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:54.424993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1de1b-c761-442e-ae3b-a5f5c6ba45d2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:54.438265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:54.438958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:54.439246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:54.439526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:54.440308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:54.440779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:54.442293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:54.459712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 80/169] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:54.460776 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:54.484400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4811b-cc34-4ec0-b6a9-662b26625e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:54.485462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4811b-cc34-4ec0-b6a9-662b26625e5c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:54.485808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4811b-cc34-4ec0-b6a9-662b26625e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:54.486150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4811b-cc34-4ec0-b6a9-662b26625e5c None admin] PATH_INFO: `/v3/users/156da27655884b45a9652caa634889b2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:54.487360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4811b-cc34-4ec0-b6a9-662b26625e5c None admin] RBAC: Authorizing `identity:get_user(user_id=156da27655884b45a9652caa634889b2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:54.489095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4811b-cc34-4ec0-b6a9-662b26625e5c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:54.490751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 90/170] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:54 2022] GET /identity/v3/users/156da27655884b45a9652caa634889b2 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:54.496489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4811b-cc34-4ec0-b6a9-662b26625e5c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:54.561005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:54.668247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64c4a7-7adb-4954-a1cc-4fd27e2b768a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:54.686516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4811b-cc34-4ec0-b6a9-662b26625e5c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:54.737013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:54.748982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7994eb-8e66-488f-817d-3bbeae84a6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:54.749877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7994eb-8e66-488f-817d-3bbeae84a6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:54.750167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7994eb-8e66-488f-817d-3bbeae84a6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:54.750449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7994eb-8e66-488f-817d-3bbeae84a6a9 None admin] PATH_INFO: `/v3/users/a8ba729f356d47049a450132ac3851da` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:54.756915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7994eb-8e66-488f-817d-3bbeae84a6a9 None admin] RBAC: Authorizing `identity:get_user(user_id=a8ba729f356d47049a450132ac3851da)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:54.758472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7994eb-8e66-488f-817d-3bbeae84a6a9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:54.760380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8cafb-2b98-4dca-941b-b8bdd7be0ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:54.760835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 91/171] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:54 2022] GET /identity/v3/users/a8ba729f356d47049a450132ac3851da => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:54.761783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7994eb-8e66-488f-817d-3bbeae84a6a9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:54.762318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8cafb-2b98-4dca-941b-b8bdd7be0ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:54.762318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8cafb-2b98-4dca-941b-b8bdd7be0ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:54.762509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8cafb-2b98-4dca-941b-b8bdd7be0ab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:54.890786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8cafb-2b98-4dca-941b-b8bdd7be0ab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:54.892810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8cafb-2b98-4dca-941b-b8bdd7be0ab8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:54.922151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7994eb-8e66-488f-817d-3bbeae84a6a9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:54.930339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:54.960743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d947a-1d3c-4909-a439-e0d110ed5ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:54.961789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d947a-1d3c-4909-a439-e0d110ed5ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:54.962278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d947a-1d3c-4909-a439-e0d110ed5ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:54.963426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d947a-1d3c-4909-a439-e0d110ed5ea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.041197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d947a-1d3c-4909-a439-e0d110ed5ea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:55.043349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d947a-1d3c-4909-a439-e0d110ed5ea8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:55.142403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 81/172] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:55.144194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8cafb-2b98-4dca-941b-b8bdd7be0ab8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:55.175421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 92/173] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:55.176645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d947a-1d3c-4909-a439-e0d110ed5ea8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:55.190636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:55.193858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:55.371209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d947a-1d3c-4909-a439-e0d110ed5ea8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:55.390744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8cafb-2b98-4dca-941b-b8bdd7be0ab8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:55.416808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd8e60-26cd-48f5-9a1f-5e31ce901314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:55.417960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd8e60-26cd-48f5-9a1f-5e31ce901314 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.418398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd8e60-26cd-48f5-9a1f-5e31ce901314 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.418816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd8e60-26cd-48f5-9a1f-5e31ce901314 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.420231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd8e60-26cd-48f5-9a1f-5e31ce901314 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:55.421870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd8e60-26cd-48f5-9a1f-5e31ce901314 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:55.424095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 93/174] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:55 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:55.425099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd8e60-26cd-48f5-9a1f-5e31ce901314 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:55.450693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4d80f-fc89-45d7-9a62-0d96944111ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:55.451961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4d80f-fc89-45d7-9a62-0d96944111ef None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.452275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4d80f-fc89-45d7-9a62-0d96944111ef None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.452589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4d80f-fc89-45d7-9a62-0d96944111ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.530279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:55.555927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4d80f-fc89-45d7-9a62-0d96944111ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:55.557787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4d80f-fc89-45d7-9a62-0d96944111ef None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:55.560316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd8e60-26cd-48f5-9a1f-5e31ce901314 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:55.587151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:55.610658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512a191-2e6f-4a6b-b41a-4d3917f6c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:55.611630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a191-2e6f-4a6b-b41a-4d3917f6c28f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.612085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a191-2e6f-4a6b-b41a-4d3917f6c28f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.612559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512a191-2e6f-4a6b-b41a-4d3917f6c28f None admin] PATH_INFO: `/v3/projects/5e12fbee5aaf4b70a8f4b54dc8c9e8f2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.613962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512a191-2e6f-4a6b-b41a-4d3917f6c28f None admin] RBAC: Authorizing `identity:get_project(project_id=5e12fbee5aaf4b70a8f4b54dc8c9e8f2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:55.615439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512a191-2e6f-4a6b-b41a-4d3917f6c28f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:55.617219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 94/175] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:55 2022] GET /identity/v3/projects/5e12fbee5aaf4b70a8f4b54dc8c9e8f2 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:55.620678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae128a7-78e8-4ed4-9b4b-b34540f62f55 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.626686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae128a7-78e8-4ed4-9b4b-b34540f62f55 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.626932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae128a7-78e8-4ed4-9b4b-b34540f62f55 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.628441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 95/176] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:55.633185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5201d-16e2-478b-bb2f-7abc302b6ae6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.633185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5201d-16e2-478b-bb2f-7abc302b6ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.633185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5201d-16e2-478b-bb2f-7abc302b6ae6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.633185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 96/177] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:55.633185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5201d-16e2-478b-bb2f-7abc302b6ae6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:55.709641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 82/178] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:55.711281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ec66b-6a00-4b20-a2ab-c2d569e6b8e0 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.711574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ec66b-6a00-4b20-a2ab-c2d569e6b8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.711854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ec66b-6a00-4b20-a2ab-c2d569e6b8e0 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.714047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 83/179] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:55.715151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ec66b-6a00-4b20-a2ab-c2d569e6b8e0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:55.794359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5201d-16e2-478b-bb2f-7abc302b6ae6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:55.832298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0cd3f-0bdb-404a-8688-0dfad837d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:55.832977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0cd3f-0bdb-404a-8688-0dfad837d7ee None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.833257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0cd3f-0bdb-404a-8688-0dfad837d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.833531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0cd3f-0bdb-404a-8688-0dfad837d7ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.834367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0cd3f-0bdb-404a-8688-0dfad837d7ee None admin] RBAC: Adding query filter params (role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7, user.id=156da27655884b45a9652caa634889b2) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:55.834480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ec66b-6a00-4b20-a2ab-c2d569e6b8e0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:55.834675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0cd3f-0bdb-404a-8688-0dfad837d7ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:55.836137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0cd3f-0bdb-404a-8688-0dfad837d7ee None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:55.842794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 97/180] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:55 2022] GET /identity/v3/role_assignments?user.id=156da27655884b45a9652caa634889b2&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:55.844093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af38c7-69db-4614-b7cd-1322fac0ce36 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.844410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af38c7-69db-4614-b7cd-1322fac0ce36 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.844691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af38c7-69db-4614-b7cd-1322fac0ce36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.858482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:55.865778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-16af38c7-69db-4614-b7cd-1322fac0ce36 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:55.867160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-16af38c7-69db-4614-b7cd-1322fac0ce36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:55.867787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:55.892341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883041a-76f4-442d-b020-61a80dbe54fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:55.892977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883041a-76f4-442d-b020-61a80dbe54fc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.893286 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883041a-76f4-442d-b020-61a80dbe54fc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.893547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883041a-76f4-442d-b020-61a80dbe54fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.894410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883041a-76f4-442d-b020-61a80dbe54fc None admin] RBAC: Adding query filter params (role.id=932e429fa28248f285663a90fc880876, scope.project.id=5e12fbee5aaf4b70a8f4b54dc8c9e8f2, user.id=156da27655884b45a9652caa634889b2) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:55.894744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883041a-76f4-442d-b020-61a80dbe54fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:55.896290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883041a-76f4-442d-b020-61a80dbe54fc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:55.912292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 84/181] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:55 2022] GET /identity/v3/role_assignments?user.id=156da27655884b45a9652caa634889b2&scope.project.id=5e12fbee5aaf4b70a8f4b54dc8c9e8f2&role.id=932e429fa28248f285663a90fc880876 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:55.912292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da218226-d10e-4c59-9a8e-30f7e6e2da1b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.912292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da218226-d10e-4c59-9a8e-30f7e6e2da1b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.912292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da218226-d10e-4c59-9a8e-30f7e6e2da1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.936188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-da218226-d10e-4c59-9a8e-30f7e6e2da1b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:55.937568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-da218226-d10e-4c59-9a8e-30f7e6e2da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:55.992456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 98/182] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:55.994781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca8fc8-9906-4647-bbbe-d02421f84bf3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:55.995283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca8fc8-9906-4647-bbbe-d02421f84bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:55.995746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca8fc8-9906-4647-bbbe-d02421f84bf3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:55.997476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 99/183] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:56.002891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca8fc8-9906-4647-bbbe-d02421f84bf3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.038853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 85/184] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:56.041203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37498b-6e52-4cec-aae0-1cd19cd5e2bd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.041203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37498b-6e52-4cec-aae0-1cd19cd5e2bd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.041203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37498b-6e52-4cec-aae0-1cd19cd5e2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.062460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dd37498b-6e52-4cec-aae0-1cd19cd5e2bd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:56.063703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dd37498b-6e52-4cec-aae0-1cd19cd5e2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:56.173399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 86/185] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:56.176149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65642-5659-4ae4-941b-58d3c0bf1f4b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.176785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65642-5659-4ae4-941b-58d3c0bf1f4b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.177216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65642-5659-4ae4-941b-58d3c0bf1f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.177597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca8fc8-9906-4647-bbbe-d02421f84bf3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:56.203740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5ab65642-5659-4ae4-941b-58d3c0bf1f4b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:56.209326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5ab65642-5659-4ae4-941b-58d3c0bf1f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:56.239180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784fa08-913d-4a07-815c-b83957540449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:56.241959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784fa08-913d-4a07-815c-b83957540449 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.242719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784fa08-913d-4a07-815c-b83957540449 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.243043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784fa08-913d-4a07-815c-b83957540449 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.247021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784fa08-913d-4a07-815c-b83957540449 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:56.250571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784fa08-913d-4a07-815c-b83957540449 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:56.253709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 100/186] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:56.254980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784fa08-913d-4a07-815c-b83957540449 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.260929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:56.274256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:56.313139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 87/187] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:56.314469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65642-5659-4ae4-941b-58d3c0bf1f4b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.397528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784fa08-913d-4a07-815c-b83957540449 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:56.427039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65642-5659-4ae4-941b-58d3c0bf1f4b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:56.431988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da89af-701f-4989-ab98-2131d73fb69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:56.432827 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da89af-701f-4989-ab98-2131d73fb69f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.433145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da89af-701f-4989-ab98-2131d73fb69f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.433394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da89af-701f-4989-ab98-2131d73fb69f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.443629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da89af-701f-4989-ab98-2131d73fb69f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-59da89af-701f-4989-ab98-2131d73fb69f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:56.452282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:56.453233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:56.454024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 101/188] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:56.454672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da89af-701f-4989-ab98-2131d73fb69f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.467533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aeab20-09ef-4eda-a335-456e5ce548f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:56.468363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aeab20-09ef-4eda-a335-456e5ce548f4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.468652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aeab20-09ef-4eda-a335-456e5ce548f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.468932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aeab20-09ef-4eda-a335-456e5ce548f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.479372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aeab20-09ef-4eda-a335-456e5ce548f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a2aeab20-09ef-4eda-a335-456e5ce548f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:56.487077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:56.488039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 88/189] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:56.489167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aeab20-09ef-4eda-a335-456e5ce548f4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.521921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:56.527815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:56.565751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da89af-701f-4989-ab98-2131d73fb69f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:56.606525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff95e2-6bc9-4755-817d-5bfdf6d99db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:56.607364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff95e2-6bc9-4755-817d-5bfdf6d99db5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.607649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff95e2-6bc9-4755-817d-5bfdf6d99db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.607926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff95e2-6bc9-4755-817d-5bfdf6d99db5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.618536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff95e2-6bc9-4755-817d-5bfdf6d99db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:56.623671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aeab20-09ef-4eda-a335-456e5ce548f4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cdff95e2-6bc9-4755-817d-5bfdf6d99db5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:56.626864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:56.627738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 102/190] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:56.628852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff95e2-6bc9-4755-817d-5bfdf6d99db5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.661330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:56.661993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.662281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.662556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.663388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7, user.id=a8ba729f356d47049a450132ac3851da) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:56.663699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:56.665188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:56.671011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 89/191] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/role_assignments?user.id=a8ba729f356d47049a450132ac3851da&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=4d21498572b14478a996ac538be5382e => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:56.671794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.747913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff95e2-6bc9-4755-817d-5bfdf6d99db5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:56.774688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:56.785702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:56.788416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f044-2d3f-43d1-93ff-fdbfeaae8d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:56.789239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f044-2d3f-43d1-93ff-fdbfeaae8d8d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.789523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a3233c-7b27-4d4e-bb5a-c98534f8f89d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:56.789633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f044-2d3f-43d1-93ff-fdbfeaae8d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.789808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f044-2d3f-43d1-93ff-fdbfeaae8d8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.791232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450f044-2d3f-43d1-93ff-fdbfeaae8d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:56.792659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450f044-2d3f-43d1-93ff-fdbfeaae8d8d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:56.794589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 103/192] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:56.795243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f044-2d3f-43d1-93ff-fdbfeaae8d8d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.838900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:56.839608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.839897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.840201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.840994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:56.841307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:56.842737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:56.856991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 90/193] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:56.856991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.921383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f044-2d3f-43d1-93ff-fdbfeaae8d8d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:56.961254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:56.961935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:56.962219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:56.962497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:56.963977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:56.964340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:56.965738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:56.974231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 104/194] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:56.974231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:56.988947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78be463-bf9f-4556-83df-b874c7608994 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.023450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.024041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.024355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.024636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.025984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:57.026299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:57.027809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:57.031861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:57.036479 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 91/195] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:57.037304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:57.061656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:57.081247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340dddc5-5816-4c28-b070-bdd0ce59af0c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.114450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565b67d-941e-4a35-8c0a-e142a2900840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.115127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b67d-941e-4a35-8c0a-e142a2900840 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.115425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b67d-941e-4a35-8c0a-e142a2900840 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.115703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b67d-941e-4a35-8c0a-e142a2900840 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.116459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565b67d-941e-4a35-8c0a-e142a2900840 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:57.117926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565b67d-941e-4a35-8c0a-e142a2900840 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:57.137993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00687d09-ccc4-419d-bb20-aedacdc72105 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.139516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 105/196] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:49:56 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:49:57.141047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565b67d-941e-4a35-8c0a-e142a2900840 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:57.181079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b359a59-2a3b-4b89-a612-5e04c7cc5c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.182895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b359a59-2a3b-4b89-a612-5e04c7cc5c87 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.183405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b359a59-2a3b-4b89-a612-5e04c7cc5c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.183920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b359a59-2a3b-4b89-a612-5e04c7cc5c87 None admin] PATH_INFO: `/v3/users/d19c405e82e44f59a1d533bc51114091` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.203982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b359a59-2a3b-4b89-a612-5e04c7cc5c87 None admin] RBAC: Authorizing `identity:get_user(user_id=d19c405e82e44f59a1d533bc51114091)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:57.205596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b359a59-2a3b-4b89-a612-5e04c7cc5c87 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:57.207728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 92/197] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:57 2022] GET /identity/v3/users/d19c405e82e44f59a1d533bc51114091 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:57.208402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b359a59-2a3b-4b89-a612-5e04c7cc5c87 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:57.279808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565b67d-941e-4a35-8c0a-e142a2900840 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.294913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:57.324228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcaa06-7cb8-4102-93e5-b68589d33479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.325134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcaa06-7cb8-4102-93e5-b68589d33479 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.325423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcaa06-7cb8-4102-93e5-b68589d33479 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.325702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcaa06-7cb8-4102-93e5-b68589d33479 None admin] PATH_INFO: `/v3/users/d19c405e82e44f59a1d533bc51114091` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.327004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcaa06-7cb8-4102-93e5-b68589d33479 None admin] RBAC: Authorizing `identity:get_user(user_id=d19c405e82e44f59a1d533bc51114091)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:57.327455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:57.329429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcaa06-7cb8-4102-93e5-b68589d33479 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:57.332113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 106/198] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:57 2022] GET /identity/v3/users/d19c405e82e44f59a1d533bc51114091 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:57.333018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcaa06-7cb8-4102-93e5-b68589d33479 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:57.355780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b359a59-2a3b-4b89-a612-5e04c7cc5c87 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.397677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3df8dc-d01a-4057-93a9-ca600c7e3d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.398865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3df8dc-d01a-4057-93a9-ca600c7e3d7c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.399151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3df8dc-d01a-4057-93a9-ca600c7e3d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.399433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3df8dc-d01a-4057-93a9-ca600c7e3d7c None admin] PATH_INFO: `/v3/users/dd7e8f5ac51f4e378717efe3e360bd9e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.401181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3df8dc-d01a-4057-93a9-ca600c7e3d7c None admin] RBAC: Authorizing `identity:get_user(user_id=dd7e8f5ac51f4e378717efe3e360bd9e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:57.402638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3df8dc-d01a-4057-93a9-ca600c7e3d7c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:57.404898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 93/199] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:49:57 2022] GET /identity/v3/users/dd7e8f5ac51f4e378717efe3e360bd9e => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:57.405477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3df8dc-d01a-4057-93a9-ca600c7e3d7c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:57.478710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcaa06-7cb8-4102-93e5-b68589d33479 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.523974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd01d4-0b73-443a-99b5-9be7f1f42276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.524716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd01d4-0b73-443a-99b5-9be7f1f42276 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.525644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd01d4-0b73-443a-99b5-9be7f1f42276 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.526356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd01d4-0b73-443a-99b5-9be7f1f42276 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.533345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3df8dc-d01a-4057-93a9-ca600c7e3d7c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.578088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5d8f-216c-4ebe-aeaa-53a02379637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.578899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5d8f-216c-4ebe-aeaa-53a02379637e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.579182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5d8f-216c-4ebe-aeaa-53a02379637e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.579453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5d8f-216c-4ebe-aeaa-53a02379637e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.592759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:57.610900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:57.626788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd01d4-0b73-443a-99b5-9be7f1f42276 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:57.628616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd01d4-0b73-443a-99b5-9be7f1f42276 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:57.675794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa5d8f-216c-4ebe-aeaa-53a02379637e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:57.678460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa5d8f-216c-4ebe-aeaa-53a02379637e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:57.750056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 107/200] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:57.751481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd01d4-0b73-443a-99b5-9be7f1f42276 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:57.802982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 94/201] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:57.804096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5d8f-216c-4ebe-aeaa-53a02379637e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:57.857140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:57.865290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:57.881779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd01d4-0b73-443a-99b5-9be7f1f42276 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.921658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca479ac-4651-489d-a316-e8d62fa527f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.922603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca479ac-4651-489d-a316-e8d62fa527f8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.922962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5d8f-216c-4ebe-aeaa-53a02379637e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:57.923177 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca479ac-4651-489d-a316-e8d62fa527f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.923693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca479ac-4651-489d-a316-e8d62fa527f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.961016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026e356-4d00-4487-9a1d-1279bcbd2105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:57.962037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026e356-4d00-4487-9a1d-1279bcbd2105 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:57.962423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026e356-4d00-4487-9a1d-1279bcbd2105 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:57.963023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026e356-4d00-4487-9a1d-1279bcbd2105 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:57.964433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026e356-4d00-4487-9a1d-1279bcbd2105 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:57.965991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026e356-4d00-4487-9a1d-1279bcbd2105 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:57.968122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 95/202] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:57 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:57.969636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026e356-4d00-4487-9a1d-1279bcbd2105 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:58.012190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca479ac-4651-489d-a316-e8d62fa527f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:58.014348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca479ac-4651-489d-a316-e8d62fa527f8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:58.091170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026e356-4d00-4487-9a1d-1279bcbd2105 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:58.115683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:58.117609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:58.143461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89f1e-683a-4c01-9752-e55151b76dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:58.148402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 108/203] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:49:58.148402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca479ac-4651-489d-a316-e8d62fa527f8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:58.149207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89f1e-683a-4c01-9752-e55151b76dee None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:58.149207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89f1e-683a-4c01-9752-e55151b76dee None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:58.149374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c89f1e-683a-4c01-9752-e55151b76dee None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:58.151542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c89f1e-683a-4c01-9752-e55151b76dee None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:58.153000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c89f1e-683a-4c01-9752-e55151b76dee None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:58.154880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 96/204] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:57 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:58.155575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89f1e-683a-4c01-9752-e55151b76dee None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:58.267637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c89f1e-683a-4c01-9752-e55151b76dee None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:58.284550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca479ac-4651-489d-a316-e8d62fa527f8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:58.304171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b653c6-bc31-4c66-9a48-e539fc4437b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:58.305082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b653c6-bc31-4c66-9a48-e539fc4437b3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:58.305368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b653c6-bc31-4c66-9a48-e539fc4437b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:58.305648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b653c6-bc31-4c66-9a48-e539fc4437b3 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:58.307718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b653c6-bc31-4c66-9a48-e539fc4437b3 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:58.309362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b653c6-bc31-4c66-9a48-e539fc4437b3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:58.315832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 97/205] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:49:58 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:58.315832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b653c6-bc31-4c66-9a48-e539fc4437b3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:58.337945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:58.339218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:58.339738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:58.340488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:58.342143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] RBAC: Adding query filter params (role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=5229c8d118a3441fa4e48635e621fb8d, user.id=d19c405e82e44f59a1d533bc51114091) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:58.342715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:58.344993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:58.354208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 109/206] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:58 2022] GET /identity/v3/role_assignments?user.id=d19c405e82e44f59a1d533bc51114091&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:58.355312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:58.376515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:58.422063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:58.435815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b653c6-bc31-4c66-9a48-e539fc4437b3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:58.475164 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c64fd1-7691-4446-ac21-5aea9a74a603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:58.475859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c64fd1-7691-4446-ac21-5aea9a74a603 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:58.476144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c64fd1-7691-4446-ac21-5aea9a74a603 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:58.476438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c64fd1-7691-4446-ac21-5aea9a74a603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:58.477288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c64fd1-7691-4446-ac21-5aea9a74a603 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=5229c8d118a3441fa4e48635e621fb8d, user.id=d19c405e82e44f59a1d533bc51114091) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:58.477602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c64fd1-7691-4446-ac21-5aea9a74a603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:58.480047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c64fd1-7691-4446-ac21-5aea9a74a603 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:58.486951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 98/207] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:58 2022] GET /identity/v3/role_assignments?user.id=d19c405e82e44f59a1d533bc51114091&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=4d21498572b14478a996ac538be5382e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:58.503570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4a338-a08b-4268-9d2e-85ac95ae6a71 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:58.545247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a46bd-a183-4497-8f7e-6726dda01f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:58.546205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a46bd-a183-4497-8f7e-6726dda01f3c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:58.546636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a46bd-a183-4497-8f7e-6726dda01f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:58.547054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a46bd-a183-4497-8f7e-6726dda01f3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:58.548048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a46bd-a183-4497-8f7e-6726dda01f3c None admin] RBAC: Adding query filter params (role.id=c2921011b0a949eba8e918eb306cf45b, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7, user.id=dd7e8f5ac51f4e378717efe3e360bd9e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:49:58.548534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a46bd-a183-4497-8f7e-6726dda01f3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:49:58.550253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a46bd-a183-4497-8f7e-6726dda01f3c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:49:58.558056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 110/208] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:49:58 2022] GET /identity/v3/role_assignments?user.id=dd7e8f5ac51f4e378717efe3e360bd9e&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=c2921011b0a949eba8e918eb306cf45b => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:49:59.617374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f33866-d030-4225-a2f4-1e3cdeb82cb7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:59.617866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f33866-d030-4225-a2f4-1e3cdeb82cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:59.617977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f33866-d030-4225-a2f4-1e3cdeb82cb7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:59.620037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 99/209] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:59.626133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491afcd-ebce-4d04-8500-978559b127eb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:59.626568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491afcd-ebce-4d04-8500-978559b127eb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:59.626821 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491afcd-ebce-4d04-8500-978559b127eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:59.644420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9491afcd-ebce-4d04-8500-978559b127eb None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:59.645629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9491afcd-ebce-4d04-8500-978559b127eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:59.718063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 111/210] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:49:59.741304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f33866-d030-4225-a2f4-1e3cdeb82cb7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:59.858521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f33866-d030-4225-a2f4-1e3cdeb82cb7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:49:59.878844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:49:59.899050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185c9d5-5d05-4619-b2b3-e261ed368977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:49:59.900982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185c9d5-5d05-4619-b2b3-e261ed368977 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:59.901702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1a2b2-0986-4e44-8d3a-b46d587d5241 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:59.901929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185c9d5-5d05-4619-b2b3-e261ed368977 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:59.902196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1a2b2-0986-4e44-8d3a-b46d587d5241 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:59.902551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1a2b2-0986-4e44-8d3a-b46d587d5241 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:59.902745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185c9d5-5d05-4619-b2b3-e261ed368977 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:59.904084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 112/211] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:49:59.909756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2145516-1242-4161-b1db-fc645027142a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:49:59.910102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2145516-1242-4161-b1db-fc645027142a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:49:59.910459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2145516-1242-4161-b1db-fc645027142a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:49:59.915318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185c9d5-5d05-4619-b2b3-e261ed368977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c185c9d5-5d05-4619-b2b3-e261ed368977 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:49:59.925601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:49:59.926515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:49:59.927293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:49:59.929084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e2145516-1242-4161-b1db-fc645027142a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:49:59.929846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 100/212] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:49:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:49:59.930665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e2145516-1242-4161-b1db-fc645027142a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:49:59.938469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185c9d5-5d05-4619-b2b3-e261ed368977 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:49:59.969383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:00.024051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 113/213] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:00.049030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2145516-1242-4161-b1db-fc645027142a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:00.068365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185c9d5-5d05-4619-b2b3-e261ed368977 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:00.106428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:00.107528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:00.108006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:00.108648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:00.109593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:00.110095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:00.111985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:00.130320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 101/214] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:49:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:00.131645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:00.174391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2145516-1242-4161-b1db-fc645027142a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:00.180840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:00.214729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58617de2-c3ae-4b39-8d79-294e4f8e6c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:00.215589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58617de2-c3ae-4b39-8d79-294e4f8e6c21 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:00.215878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58617de2-c3ae-4b39-8d79-294e4f8e6c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:00.216227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58617de2-c3ae-4b39-8d79-294e4f8e6c21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:00.217474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58617de2-c3ae-4b39-8d79-294e4f8e6c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:00.218944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58617de2-c3ae-4b39-8d79-294e4f8e6c21 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:00.220914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 114/215] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:00.226656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58617de2-c3ae-4b39-8d79-294e4f8e6c21 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:00.249985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa473ea-f872-4371-923e-14e83bb044ee None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:00.264437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:00.290431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e922721-a3e9-4075-9e84-cbd2924517e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:00.292196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e922721-a3e9-4075-9e84-cbd2924517e9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:00.292793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e922721-a3e9-4075-9e84-cbd2924517e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:00.293320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e922721-a3e9-4075-9e84-cbd2924517e9 None admin] PATH_INFO: `/v3/users/49d5a4807e5045208cb5c45d64973f41` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:00.309979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e922721-a3e9-4075-9e84-cbd2924517e9 None admin] RBAC: Authorizing `identity:get_user(user_id=49d5a4807e5045208cb5c45d64973f41)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:00.311893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e922721-a3e9-4075-9e84-cbd2924517e9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:00.318653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 102/216] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:00 2022] GET /identity/v3/users/49d5a4807e5045208cb5c45d64973f41 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:00.319731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e922721-a3e9-4075-9e84-cbd2924517e9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:00.336669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58617de2-c3ae-4b39-8d79-294e4f8e6c21 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:00.375357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5597c-5d39-4888-937d-8523fc7c2a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:00.375963 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5597c-5d39-4888-937d-8523fc7c2a5c None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:00.376283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5597c-5d39-4888-937d-8523fc7c2a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:00.376570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5597c-5d39-4888-937d-8523fc7c2a5c None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:00.377289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5597c-5d39-4888-937d-8523fc7c2a5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:00.379341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5597c-5d39-4888-937d-8523fc7c2a5c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:00.412361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 115/217] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:50:00 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:50:00.433339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:00.437998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e922721-a3e9-4075-9e84-cbd2924517e9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:00.479730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb36819-fedd-4e1f-abbb-d26d320cbef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:00.480895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb36819-fedd-4e1f-abbb-d26d320cbef3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:00.481381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb36819-fedd-4e1f-abbb-d26d320cbef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:00.481899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb36819-fedd-4e1f-abbb-d26d320cbef3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:00.562400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb36819-fedd-4e1f-abbb-d26d320cbef3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:00.564313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb36819-fedd-4e1f-abbb-d26d320cbef3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:00.680732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:00.690993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 103/218] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:00.691773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5597c-5d39-4888-937d-8523fc7c2a5c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:00.803398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:00.817657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5597c-5d39-4888-937d-8523fc7c2a5c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:00.857672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba323a-d296-4b6b-bd02-e62f8528d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:00.858555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba323a-d296-4b6b-bd02-e62f8528d864 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:00.858842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba323a-d296-4b6b-bd02-e62f8528d864 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:00.859123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba323a-d296-4b6b-bd02-e62f8528d864 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:00.860359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba323a-d296-4b6b-bd02-e62f8528d864 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:00.861847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba323a-d296-4b6b-bd02-e62f8528d864 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:00.863943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 116/219] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:00 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:00.870109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb36819-fedd-4e1f-abbb-d26d320cbef3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:00.934542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365c8d-6c0e-4c9d-9fb6-5b8facfd5cc6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:00.935170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365c8d-6c0e-4c9d-9fb6-5b8facfd5cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:00.935639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea365c8d-6c0e-4c9d-9fb6-5b8facfd5cc6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:00.937750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 117/220] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:50:00.943251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81075c-ce30-4c77-9cb0-151e2191af72 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:00.943544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81075c-ce30-4c77-9cb0-151e2191af72 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:00.943828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81075c-ce30-4c77-9cb0-151e2191af72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:00.963072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0d81075c-ce30-4c77-9cb0-151e2191af72 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:00.964297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0d81075c-ce30-4c77-9cb0-151e2191af72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:01.001988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb36819-fedd-4e1f-abbb-d26d320cbef3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:01.041627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:01.042538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.042774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.043053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.044099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] RBAC: Adding query filter params (role.id=932e429fa28248f285663a90fc880876, scope.project.id=5229c8d118a3441fa4e48635e621fb8d, user.id=49d5a4807e5045208cb5c45d64973f41) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:01.044225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:01.045753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:01.050141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 118/221] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:01.051755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea21e8-7fc5-4055-bc63-3bbf797ca8ca None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.052267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea21e8-7fc5-4055-bc63-3bbf797ca8ca None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.052738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea21e8-7fc5-4055-bc63-3bbf797ca8ca None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.055253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 119/222] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:01.059259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 104/223] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:00 2022] GET /identity/v3/role_assignments?user.id=49d5a4807e5045208cb5c45d64973f41&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=932e429fa28248f285663a90fc880876 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:01.060938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43fb5-a2c0-424c-a0ca-df48af489553 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.061434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43fb5-a2c0-424c-a0ca-df48af489553 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.061911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb43fb5-a2c0-424c-a0ca-df48af489553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.076272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:01.101321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aeb43fb5-a2c0-424c-a0ca-df48af489553 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:01.102852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aeb43fb5-a2c0-424c-a0ca-df48af489553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:01.167696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:01.216934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 120/224] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:01.244561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb43fb5-a2c0-424c-a0ca-df48af489553 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:01.251668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c1e0e-a851-4474-8836-790674038982 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:01.273067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:01.315740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b71a84-f893-4ff1-9921-e00ed0527dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:01.317131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b71a84-f893-4ff1-9921-e00ed0527dba None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.317633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b71a84-f893-4ff1-9921-e00ed0527dba None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.318125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b71a84-f893-4ff1-9921-e00ed0527dba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.333243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b71a84-f893-4ff1-9921-e00ed0527dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f8b71a84-f893-4ff1-9921-e00ed0527dba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:01.341959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:01.342865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:01.343645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 105/225] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:50:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:01.348947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b71a84-f893-4ff1-9921-e00ed0527dba None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:01.387592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb43fb5-a2c0-424c-a0ca-df48af489553 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:01.423219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a4a0-b4fc-4d25-9efc-a2f76e820e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:01.424498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a4a0-b4fc-4d25-9efc-a2f76e820e5f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.424995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a4a0-b4fc-4d25-9efc-a2f76e820e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.425491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a4a0-b4fc-4d25-9efc-a2f76e820e5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.436745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a4a0-b4fc-4d25-9efc-a2f76e820e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8d81a4a0-b4fc-4d25-9efc-a2f76e820e5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:01.445118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:01.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:01.447370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:01.450054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 121/226] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:01.455762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a4a0-b4fc-4d25-9efc-a2f76e820e5f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:01.544705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:01.564079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a4a0-b4fc-4d25-9efc-a2f76e820e5f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:01.627841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:01.628618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b71a84-f893-4ff1-9921-e00ed0527dba None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:01.629181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.629656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.630380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.632311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:01.632853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:01.635126 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:01.652511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 122/227] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:01.654883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:01.676714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:01.700976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:01.701762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.701957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.702241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.703017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:01.703333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:01.704763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:01.725788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 106/228] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:01.732710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:01.833774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414312d-8da4-4a57-bf2e-99281dd071fa None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:01.849137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854fed-21a1-43b6-bb73-275d28458dec None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:01.892395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48491-35fe-461b-bcf7-aa95df39680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:01.894052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48491-35fe-461b-bcf7-aa95df39680d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.894533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48491-35fe-461b-bcf7-aa95df39680d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.895003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48491-35fe-461b-bcf7-aa95df39680d None admin] PATH_INFO: `/v3/users/d19c405e82e44f59a1d533bc51114091` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.896527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e48491-35fe-461b-bcf7-aa95df39680d None admin] RBAC: Authorizing `identity:get_user(user_id=d19c405e82e44f59a1d533bc51114091)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:01.898340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e48491-35fe-461b-bcf7-aa95df39680d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:01.900990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 107/229] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:01 2022] GET /identity/v3/users/d19c405e82e44f59a1d533bc51114091 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:01.909179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48491-35fe-461b-bcf7-aa95df39680d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:01.933454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155eed2-cd87-4547-918b-60ce822aa091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:01.934356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155eed2-cd87-4547-918b-60ce822aa091 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:01.934644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155eed2-cd87-4547-918b-60ce822aa091 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:01.934928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155eed2-cd87-4547-918b-60ce822aa091 None admin] PATH_INFO: `/v3/users/d19c405e82e44f59a1d533bc51114091` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:01.940956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155eed2-cd87-4547-918b-60ce822aa091 None admin] RBAC: Authorizing `identity:get_user(user_id=d19c405e82e44f59a1d533bc51114091)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:01.942456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155eed2-cd87-4547-918b-60ce822aa091 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:01.944182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 123/230] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:01 2022] GET /identity/v3/users/d19c405e82e44f59a1d533bc51114091 => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:01.951035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155eed2-cd87-4547-918b-60ce822aa091 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:01.975752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:01.976420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:02.075423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e48491-35fe-461b-bcf7-aa95df39680d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:02.143083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7189f-366d-496f-b52d-bd1e5fd94e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:02.143893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7189f-366d-496f-b52d-bd1e5fd94e52 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:02.144299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7189f-366d-496f-b52d-bd1e5fd94e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:02.144589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7189f-366d-496f-b52d-bd1e5fd94e52 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:02.147872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155eed2-cd87-4547-918b-60ce822aa091 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:02.148898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7189f-366d-496f-b52d-bd1e5fd94e52 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:02.150744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7189f-366d-496f-b52d-bd1e5fd94e52 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:02.152747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 108/231] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:01 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:02.158394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7189f-366d-496f-b52d-bd1e5fd94e52 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:02.187686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad43d8e-5ceb-4c12-bc85-258d434ac7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:02.189853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad43d8e-5ceb-4c12-bc85-258d434ac7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:02.190848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad43d8e-5ceb-4c12-bc85-258d434ac7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:02.191143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad43d8e-5ceb-4c12-bc85-258d434ac7e9 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:02.194001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad43d8e-5ceb-4c12-bc85-258d434ac7e9 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:02.195479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad43d8e-5ceb-4c12-bc85-258d434ac7e9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:02.197951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 124/232] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:01 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:02.209131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad43d8e-5ceb-4c12-bc85-258d434ac7e9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:02.295363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7189f-366d-496f-b52d-bd1e5fd94e52 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:02.330511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:02.343309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cfab4-9f04-4c66-8c77-5e006d196e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:02.344254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfab4-9f04-4c66-8c77-5e006d196e68 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:02.344583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfab4-9f04-4c66-8c77-5e006d196e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:02.345224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfab4-9f04-4c66-8c77-5e006d196e68 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d/users/d19c405e82e44f59a1d533bc51114091/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:02.350647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cfab4-9f04-4c66-8c77-5e006d196e68 None admin] RBAC: Authorizing `identity:create_grant(project_id=5229c8d118a3441fa4e48635e621fb8d, user_id=d19c405e82e44f59a1d533bc51114091, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:02.353505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cfab4-9f04-4c66-8c77-5e006d196e68 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:02.377599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 109/233] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:02 2022] PUT /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d/users/d19c405e82e44f59a1d533bc51114091/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 11:50:02.412704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:02.448138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad43d8e-5ceb-4c12-bc85-258d434ac7e9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:02.502007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68486d45-c241-47ad-b45e-e4ac02adb834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:02.502936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68486d45-c241-47ad-b45e-e4ac02adb834 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:02.503220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68486d45-c241-47ad-b45e-e4ac02adb834 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:02.503500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68486d45-c241-47ad-b45e-e4ac02adb834 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d/users/d19c405e82e44f59a1d533bc51114091/roles/a511e28f566a49b4bacfbd0325aa44ed` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:02.505866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68486d45-c241-47ad-b45e-e4ac02adb834 None admin] RBAC: Authorizing `identity:create_grant(project_id=5229c8d118a3441fa4e48635e621fb8d, user_id=d19c405e82e44f59a1d533bc51114091, role_id=a511e28f566a49b4bacfbd0325aa44ed)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:02.507367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68486d45-c241-47ad-b45e-e4ac02adb834 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:02.527796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 125/234] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:02 2022] PUT /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d/users/d19c405e82e44f59a1d533bc51114091/roles/a511e28f566a49b4bacfbd0325aa44ed => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:03.245345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e75924-54d8-4081-b6ee-1b155696ac23 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:03.245888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e75924-54d8-4081-b6ee-1b155696ac23 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:03.245888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e75924-54d8-4081-b6ee-1b155696ac23 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:03.247581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 110/235] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:03.252715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f27da-19c3-4601-8974-710e82f03407 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:03.253125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f27da-19c3-4601-8974-710e82f03407 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:03.253330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f27da-19c3-4601-8974-710e82f03407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:03.271956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-116f27da-19c3-4601-8974-710e82f03407 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:03.273190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-116f27da-19c3-4601-8974-710e82f03407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:03.370114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 126/236] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:03.393775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e75924-54d8-4081-b6ee-1b155696ac23 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:03.462207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c595-1b96-4e51-ad3d-18700e75e5b9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:03.463063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c595-1b96-4e51-ad3d-18700e75e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:03.463467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c595-1b96-4e51-ad3d-18700e75e5b9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:03.472188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 127/237] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:03.476463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127d6e6-e933-4f6f-9abf-df6d6f501df2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:03.476745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127d6e6-e933-4f6f-9abf-df6d6f501df2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:03.477043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127d6e6-e933-4f6f-9abf-df6d6f501df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:03.503151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b127d6e6-e933-4f6f-9abf-df6d6f501df2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:03.505765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b127d6e6-e933-4f6f-9abf-df6d6f501df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:03.536096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e75924-54d8-4081-b6ee-1b155696ac23 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:03.600452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f525d2-76c1-48a8-9343-2faaadf86383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:03.603178 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f525d2-76c1-48a8-9343-2faaadf86383 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:03.603574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f525d2-76c1-48a8-9343-2faaadf86383 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:03.606114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f525d2-76c1-48a8-9343-2faaadf86383 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:03.630580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f525d2-76c1-48a8-9343-2faaadf86383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:03.644538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c8f525d2-76c1-48a8-9343-2faaadf86383 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:03.645012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:03.645876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:03.646672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:03.648722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 128/238] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:03.650693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 111/239] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:03.656327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127d6e6-e933-4f6f-9abf-df6d6f501df2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:03.698785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f525d2-76c1-48a8-9343-2faaadf86383 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:03.769443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:03.855805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127d6e6-e933-4f6f-9abf-df6d6f501df2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:03.886350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f525d2-76c1-48a8-9343-2faaadf86383 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:03.895679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:03.896389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:03.896673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:03.896947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:03.897696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:03.898003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:03.899391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:03.908980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 129/240] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:03.914939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.000202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb35b92-d7d0-4a1d-9b97-751220a21065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:04.001407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb35b92-d7d0-4a1d-9b97-751220a21065 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.001904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb35b92-d7d0-4a1d-9b97-751220a21065 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.002381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb35b92-d7d0-4a1d-9b97-751220a21065 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.004255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb35b92-d7d0-4a1d-9b97-751220a21065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:04.006330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb35b92-d7d0-4a1d-9b97-751220a21065 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:04.008905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 112/241] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:04.010995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658a16c-e0ed-4329-93a3-8347ffc2d64f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.011290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:04.011758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658a16c-e0ed-4329-93a3-8347ffc2d64f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.012279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658a16c-e0ed-4329-93a3-8347ffc2d64f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.018608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 113/242] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:04.020715 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658a16c-e0ed-4329-93a3-8347ffc2d64f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.045307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c79bbe-bf9d-4fd4-8837-69a50da9ed11 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:04.097134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3708a-7800-451d-8d0f-968f5d87846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:04.098089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3708a-7800-451d-8d0f-968f5d87846d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.098405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3708a-7800-451d-8d0f-968f5d87846d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.098713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3708a-7800-451d-8d0f-968f5d87846d None admin] PATH_INFO: `/v3/users/c2893c458cea4aaba7e0b0969aa0040a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.114415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3708a-7800-451d-8d0f-968f5d87846d None admin] RBAC: Authorizing `identity:get_user(user_id=c2893c458cea4aaba7e0b0969aa0040a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:04.115960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3708a-7800-451d-8d0f-968f5d87846d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:04.118171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 130/243] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:03 2022] GET /identity/v3/users/c2893c458cea4aaba7e0b0969aa0040a => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:04.120131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c399b7c-a20b-4f89-b0e0-9ea682cdf8fb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.120450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c399b7c-a20b-4f89-b0e0-9ea682cdf8fb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.120740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c399b7c-a20b-4f89-b0e0-9ea682cdf8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.138655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2c399b7c-a20b-4f89-b0e0-9ea682cdf8fb None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:04.139660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2c399b7c-a20b-4f89-b0e0-9ea682cdf8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:04.153481 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658a16c-e0ed-4329-93a3-8347ffc2d64f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:04.163498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:04.191727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e71e4-548e-47ec-ae43-6617dba4b5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:04.192810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e71e4-548e-47ec-ae43-6617dba4b5db None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.193295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e71e4-548e-47ec-ae43-6617dba4b5db None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.193767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e71e4-548e-47ec-ae43-6617dba4b5db None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.194681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e71e4-548e-47ec-ae43-6617dba4b5db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:04.196446 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e71e4-548e-47ec-ae43-6617dba4b5db None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:04.224180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 131/244] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:04.224180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c399b7c-a20b-4f89-b0e0-9ea682cdf8fb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.228612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 114/245] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:50:04 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:04.246659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e71e4-548e-47ec-ae43-6617dba4b5db None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.281470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:04.348497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c399b7c-a20b-4f89-b0e0-9ea682cdf8fb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:04.391944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e71e4-548e-47ec-ae43-6617dba4b5db None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:04.397108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c60790-3c8c-4d7f-8371-1ea831865b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:04.398070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c60790-3c8c-4d7f-8371-1ea831865b26 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.398561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c60790-3c8c-4d7f-8371-1ea831865b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.399013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c60790-3c8c-4d7f-8371-1ea831865b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.432513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480873-bd1d-4796-9266-17bc12666299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:04.433448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480873-bd1d-4796-9266-17bc12666299 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.433810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480873-bd1d-4796-9266-17bc12666299 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.434159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480873-bd1d-4796-9266-17bc12666299 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.447519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90480873-bd1d-4796-9266-17bc12666299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-90480873-bd1d-4796-9266-17bc12666299 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:04.456338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:04.457756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:04.458587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 115/246] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:04.467937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480873-bd1d-4796-9266-17bc12666299 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.507452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c60790-3c8c-4d7f-8371-1ea831865b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:04.514187 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c60790-3c8c-4d7f-8371-1ea831865b26 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:04.532818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:04.593456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:04.630969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480873-bd1d-4796-9266-17bc12666299 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:04.669562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 132/247] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:04.669873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:04.670568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.670856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.671135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.671905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:04.675870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:04.675870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:04.677220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c60790-3c8c-4d7f-8371-1ea831865b26 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.684223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 116/248] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:04.691619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.811756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:04.820232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c60790-3c8c-4d7f-8371-1ea831865b26 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:04.836003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e364c-f596-49bc-b658-71863bb9f52f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:04.871173 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f59093-07c5-4a2b-bdd3-d185de5ead58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:04.873147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f59093-07c5-4a2b-bdd3-d185de5ead58 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.873638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f59093-07c5-4a2b-bdd3-d185de5ead58 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.874323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f59093-07c5-4a2b-bdd3-d185de5ead58 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.876545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f59093-07c5-4a2b-bdd3-d185de5ead58 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:04.878776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f59093-07c5-4a2b-bdd3-d185de5ead58 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:04.880728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc875841-6ddd-45d0-a8d7-4fa42d0d9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:04.881611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 133/249] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:04 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:04.885176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875841-6ddd-45d0-a8d7-4fa42d0d9536 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:04.885721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875841-6ddd-45d0-a8d7-4fa42d0d9536 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:04.886221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875841-6ddd-45d0-a8d7-4fa42d0d9536 None admin] PATH_INFO: `/v3/users/49d5a4807e5045208cb5c45d64973f41` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:04.887815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc875841-6ddd-45d0-a8d7-4fa42d0d9536 None admin] RBAC: Authorizing `identity:get_user(user_id=49d5a4807e5045208cb5c45d64973f41)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:04.889538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc875841-6ddd-45d0-a8d7-4fa42d0d9536 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:04.891479 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 117/250] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:04 2022] GET /identity/v3/users/49d5a4807e5045208cb5c45d64973f41 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:04.893864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f59093-07c5-4a2b-bdd3-d185de5ead58 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.898044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc875841-6ddd-45d0-a8d7-4fa42d0d9536 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:04.916321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:05.017578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f59093-07c5-4a2b-bdd3-d185de5ead58 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:05.018427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc875841-6ddd-45d0-a8d7-4fa42d0d9536 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:05.056819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ec129-70d3-469c-8c76-d751bc76724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:05.057028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4b311-b7b9-4597-9638-c874ff9e8d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:05.057564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b311-b7b9-4597-9638-c874ff9e8d36 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:05.057893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b311-b7b9-4597-9638-c874ff9e8d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:05.058175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b311-b7b9-4597-9638-c874ff9e8d36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:05.058840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ec129-70d3-469c-8c76-d751bc76724c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:05.059039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4b311-b7b9-4597-9638-c874ff9e8d36 None admin] RBAC: Adding query filter params (role.id=c2921011b0a949eba8e918eb306cf45b, scope.project.id=5229c8d118a3441fa4e48635e621fb8d, user.id=c2893c458cea4aaba7e0b0969aa0040a) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:05.059356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4b311-b7b9-4597-9638-c874ff9e8d36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:05.059708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ec129-70d3-469c-8c76-d751bc76724c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:05.060224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ec129-70d3-469c-8c76-d751bc76724c None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:05.060914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4b311-b7b9-4597-9638-c874ff9e8d36 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:05.062246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ec129-70d3-469c-8c76-d751bc76724c None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:05.063987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ec129-70d3-469c-8c76-d751bc76724c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:05.066151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 118/251] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:04 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:05.069535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 134/252] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:04 2022] GET /identity/v3/role_assignments?user.id=c2893c458cea4aaba7e0b0969aa0040a&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=c2921011b0a949eba8e918eb306cf45b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:05.073405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ec129-70d3-469c-8c76-d751bc76724c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:05.107056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:05.205577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ec129-70d3-469c-8c76-d751bc76724c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:05.252015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065e31b-c01b-475d-94eb-f69b5752af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:05.253780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065e31b-c01b-475d-94eb-f69b5752af38 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:05.254267 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065e31b-c01b-475d-94eb-f69b5752af38 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:05.254743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065e31b-c01b-475d-94eb-f69b5752af38 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d/users/49d5a4807e5045208cb5c45d64973f41/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:05.257029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065e31b-c01b-475d-94eb-f69b5752af38 None admin] RBAC: Authorizing `identity:create_grant(project_id=5229c8d118a3441fa4e48635e621fb8d, user_id=49d5a4807e5045208cb5c45d64973f41, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:05.258708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065e31b-c01b-475d-94eb-f69b5752af38 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:05.286572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 119/253] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:05 2022] PUT /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d/users/49d5a4807e5045208cb5c45d64973f41/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:05.513663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1771e76-fbb9-45f2-b547-e78942821eeb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:05.514436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1771e76-fbb9-45f2-b547-e78942821eeb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:05.514436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1771e76-fbb9-45f2-b547-e78942821eeb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:05.516806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 135/254] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:05.524044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f201b2-2850-409a-ac6f-e5cfd9b6782d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:05.524746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f201b2-2850-409a-ac6f-e5cfd9b6782d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:05.525494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f201b2-2850-409a-ac6f-e5cfd9b6782d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:05.551033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-77f201b2-2850-409a-ac6f-e5cfd9b6782d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:05.552614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-77f201b2-2850-409a-ac6f-e5cfd9b6782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:05.581345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad7f8b-629e-4167-8939-ddf012c420fd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:05.581949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad7f8b-629e-4167-8939-ddf012c420fd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:05.582421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad7f8b-629e-4167-8939-ddf012c420fd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:05.585886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 136/255] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:05.598845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7feeec-cfe5-4c46-937d-dfedde337994 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:05.599411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7feeec-cfe5-4c46-937d-dfedde337994 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:05.599933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7feeec-cfe5-4c46-937d-dfedde337994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:05.623556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4d7feeec-cfe5-4c46-937d-dfedde337994 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:05.625661 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4d7feeec-cfe5-4c46-937d-dfedde337994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:05.674445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 120/256] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:05.714212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f201b2-2850-409a-ac6f-e5cfd9b6782d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:05.739850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 137/257] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:05.740176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:05.857440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f201b2-2850-409a-ac6f-e5cfd9b6782d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:05.861955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7feeec-cfe5-4c46-937d-dfedde337994 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:05.878905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:05.938524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20988488-16d5-4e6a-a11a-904ba618e728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:05.939362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20988488-16d5-4e6a-a11a-904ba618e728 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:05.939647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20988488-16d5-4e6a-a11a-904ba618e728 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:05.939928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20988488-16d5-4e6a-a11a-904ba618e728 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:05.961112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20988488-16d5-4e6a-a11a-904ba618e728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-20988488-16d5-4e6a-a11a-904ba618e728 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:05.971551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:05.972524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:05.973309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 121/258] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:05.979240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20988488-16d5-4e6a-a11a-904ba618e728 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:06.067397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7feeec-cfe5-4c46-937d-dfedde337994 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:06.127960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:06.135359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce06b62-9a89-45a9-8271-f39ec43d803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:06.136192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06b62-9a89-45a9-8271-f39ec43d803b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:06.136479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06b62-9a89-45a9-8271-f39ec43d803b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:06.136754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce06b62-9a89-45a9-8271-f39ec43d803b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:06.160324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce06b62-9a89-45a9-8271-f39ec43d803b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4ce06b62-9a89-45a9-8271-f39ec43d803b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:06.170251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:06.171158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:06.171937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 138/259] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:50:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:06.178247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce06b62-9a89-45a9-8271-f39ec43d803b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:06.227381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20988488-16d5-4e6a-a11a-904ba618e728 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:06.325507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:06.328893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:06.329602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:06.329892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:06.330175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:06.330947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:06.331266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:06.332712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:06.356075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 122/260] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:06.361223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:06.363828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce06b62-9a89-45a9-8271-f39ec43d803b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:06.402380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:06.403476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:06.403942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:06.404431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:06.405388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:06.405876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:06.407472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:06.421596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 139/261] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:06.423242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:06.515535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2c34d-cbea-48d6-b7de-9335e31efe6c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:06.533420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7230e-2bd1-4c9f-aff6-f4f6f1e08e52 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:06.565084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:06.571682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b0174-d1f7-492f-886c-13608fce304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:06.572973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b0174-d1f7-492f-886c-13608fce304d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:06.573452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b0174-d1f7-492f-886c-13608fce304d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:06.573910 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b0174-d1f7-492f-886c-13608fce304d None admin] PATH_INFO: `/v3/users/d19c405e82e44f59a1d533bc51114091` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:06.575370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b0174-d1f7-492f-886c-13608fce304d None admin] RBAC: Authorizing `identity:get_user(user_id=d19c405e82e44f59a1d533bc51114091)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:06.577217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b0174-d1f7-492f-886c-13608fce304d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:06.583597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 140/262] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:06 2022] GET /identity/v3/users/d19c405e82e44f59a1d533bc51114091 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:06.585026 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b0174-d1f7-492f-886c-13608fce304d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:06.588948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414bf83-216d-4e9a-8bbd-c03c3c670af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:06.590541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414bf83-216d-4e9a-8bbd-c03c3c670af9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:06.590541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414bf83-216d-4e9a-8bbd-c03c3c670af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:06.590541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414bf83-216d-4e9a-8bbd-c03c3c670af9 None admin] PATH_INFO: `/v3/users/d19c405e82e44f59a1d533bc51114091` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:06.596986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414bf83-216d-4e9a-8bbd-c03c3c670af9 None admin] RBAC: Authorizing `identity:get_user(user_id=d19c405e82e44f59a1d533bc51114091)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:06.598495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414bf83-216d-4e9a-8bbd-c03c3c670af9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:06.604190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 123/263] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:06 2022] GET /identity/v3/users/d19c405e82e44f59a1d533bc51114091 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:06.621264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414bf83-216d-4e9a-8bbd-c03c3c670af9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:06.626681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:06.748833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b0174-d1f7-492f-886c-13608fce304d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:06.801379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454139e-b798-4962-9138-5795ab5067f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:06.802475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454139e-b798-4962-9138-5795ab5067f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:06.802943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454139e-b798-4962-9138-5795ab5067f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:06.803480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454139e-b798-4962-9138-5795ab5067f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:06.812186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414bf83-216d-4e9a-8bbd-c03c3c670af9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:06.911613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52d6c3-c64e-4fed-91d4-809236a5b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:06.912312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52d6c3-c64e-4fed-91d4-809236a5b34e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:06.912599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52d6c3-c64e-4fed-91d4-809236a5b34e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:06.912884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52d6c3-c64e-4fed-91d4-809236a5b34e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:06.941643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:06.946225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454139e-b798-4962-9138-5795ab5067f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:06.948051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454139e-b798-4962-9138-5795ab5067f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:07.047953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52d6c3-c64e-4fed-91d4-809236a5b34e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:07.049494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52d6c3-c64e-4fed-91d4-809236a5b34e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:07.051500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:07.154870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 141/264] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 30 11:50:07.160895 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454139e-b798-4962-9138-5795ab5067f3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:07.197507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 124/265] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:07.206018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52d6c3-c64e-4fed-91d4-809236a5b34e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:07.347811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:07.354088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454139e-b798-4962-9138-5795ab5067f3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:07.413930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52d6c3-c64e-4fed-91d4-809236a5b34e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:07.418723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:07.421830 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070372-3c9a-4d33-93ab-9a519dde08c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:07.426388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070372-3c9a-4d33-93ab-9a519dde08c6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:07.426944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070372-3c9a-4d33-93ab-9a519dde08c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:07.427418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8070372-3c9a-4d33-93ab-9a519dde08c6 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:07.429053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070372-3c9a-4d33-93ab-9a519dde08c6 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:07.433156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8070372-3c9a-4d33-93ab-9a519dde08c6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:07.435158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 142/266] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:07 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:07.444582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070372-3c9a-4d33-93ab-9a519dde08c6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:07.481073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f110-a3af-45fa-a95f-7d7893b45afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:07.482559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f110-a3af-45fa-a95f-7d7893b45afb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:07.483352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f110-a3af-45fa-a95f-7d7893b45afb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:07.483827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9f110-a3af-45fa-a95f-7d7893b45afb None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:07.485330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9f110-a3af-45fa-a95f-7d7893b45afb None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:07.487453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9f110-a3af-45fa-a95f-7d7893b45afb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:07.489809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 125/267] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:07 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:07.496538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f110-a3af-45fa-a95f-7d7893b45afb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:07.663052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8070372-3c9a-4d33-93ab-9a519dde08c6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:07.672407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9f110-a3af-45fa-a95f-7d7893b45afb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:07.711194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe6db3-f67b-475e-9e3a-c6e9d1539f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:07.711853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6db3-f67b-475e-9e3a-c6e9d1539f42 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:07.712899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6db3-f67b-475e-9e3a-c6e9d1539f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:07.713339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6db3-f67b-475e-9e3a-c6e9d1539f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:07.714328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe6db3-f67b-475e-9e3a-c6e9d1539f42 None admin] RBAC: Adding query filter params (role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=5229c8d118a3441fa4e48635e621fb8d, user.id=d19c405e82e44f59a1d533bc51114091) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:07.714721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe6db3-f67b-475e-9e3a-c6e9d1539f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:07.716309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe6db3-f67b-475e-9e3a-c6e9d1539f42 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:07.723136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 126/268] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:07 2022] GET /identity/v3/role_assignments?user.id=d19c405e82e44f59a1d533bc51114091&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:07.730192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c208fb-0b3c-4b18-a53d-321ae3b786f3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:07.730697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c208fb-0b3c-4b18-a53d-321ae3b786f3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:07.731140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c208fb-0b3c-4b18-a53d-321ae3b786f3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:07.733068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 127/269] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:07.741434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabf06e-f95f-440e-8f3a-215e111d1485 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:07.742040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabf06e-f95f-440e-8f3a-215e111d1485 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:07.742513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabf06e-f95f-440e-8f3a-215e111d1485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:07.764952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:07.765650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:07.765944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:07.766238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:07.767350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=5229c8d118a3441fa4e48635e621fb8d, user.id=d19c405e82e44f59a1d533bc51114091) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:07.767676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:07.769203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:07.777060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fcabf06e-f95f-440e-8f3a-215e111d1485 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:07.778575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fcabf06e-f95f-440e-8f3a-215e111d1485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:07.786952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 143/270] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:07 2022] GET /identity/v3/role_assignments?user.id=d19c405e82e44f59a1d533bc51114091&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=4d21498572b14478a996ac538be5382e => generated 608 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:07.830476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:07.898828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 128/271] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:07.936574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:07.992227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:08.098893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90aba54-cb37-4766-aff6-a546befdfaed None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:08.146381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19982e5d-56ee-40ad-833e-18a2e3c24598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:08.147218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19982e5d-56ee-40ad-833e-18a2e3c24598 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.147506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19982e5d-56ee-40ad-833e-18a2e3c24598 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.147786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19982e5d-56ee-40ad-833e-18a2e3c24598 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.167130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19982e5d-56ee-40ad-833e-18a2e3c24598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-19982e5d-56ee-40ad-833e-18a2e3c24598 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:08.175144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:08.176095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:08.176930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 144/272] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:08.185659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabf06e-f95f-440e-8f3a-215e111d1485 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:08.329486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabf06e-f95f-440e-8f3a-215e111d1485 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:08.374965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4b6ae-9545-46be-b0a7-ab12453972c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:08.375646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4b6ae-9545-46be-b0a7-ab12453972c8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.375928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4b6ae-9545-46be-b0a7-ab12453972c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.376305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4b6ae-9545-46be-b0a7-ab12453972c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.377065 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4b6ae-9545-46be-b0a7-ab12453972c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:08.377372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4b6ae-9545-46be-b0a7-ab12453972c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:08.378768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4b6ae-9545-46be-b0a7-ab12453972c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:08.391271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 129/273] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:08.397360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19982e5d-56ee-40ad-833e-18a2e3c24598 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:08.502883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3def12-e078-46eb-8094-5a0414046904 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.503770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3def12-e078-46eb-8094-5a0414046904 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.504410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3def12-e078-46eb-8094-5a0414046904 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.505960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 130/274] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:08.512275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80918dbf-cbb2-4c35-9709-9ccea675df48 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.512563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80918dbf-cbb2-4c35-9709-9ccea675df48 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.512838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80918dbf-cbb2-4c35-9709-9ccea675df48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.530744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:08.540882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19982e5d-56ee-40ad-833e-18a2e3c24598 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:08.545415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-80918dbf-cbb2-4c35-9709-9ccea675df48 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:08.546624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-80918dbf-cbb2-4c35-9709-9ccea675df48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:08.570132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:08.585071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04196911-0ab2-433e-9f90-5439b9b2f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:08.586559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04196911-0ab2-433e-9f90-5439b9b2f620 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.587038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04196911-0ab2-433e-9f90-5439b9b2f620 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.588096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04196911-0ab2-433e-9f90-5439b9b2f620 None admin] PATH_INFO: `/v3/users/8a1bfd7582fd426f9461d949b07efc50` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.607710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04196911-0ab2-433e-9f90-5439b9b2f620 None admin] RBAC: Authorizing `identity:get_user(user_id=8a1bfd7582fd426f9461d949b07efc50)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:08.610126 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04196911-0ab2-433e-9f90-5439b9b2f620 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:08.612521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 145/275] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:08 2022] GET /identity/v3/users/8a1bfd7582fd426f9461d949b07efc50 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:08.618865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04196911-0ab2-433e-9f90-5439b9b2f620 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:08.632742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 131/276] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:08.657053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80918dbf-cbb2-4c35-9709-9ccea675df48 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:08.739139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04196911-0ab2-433e-9f90-5439b9b2f620 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:08.761755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80918dbf-cbb2-4c35-9709-9ccea675df48 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:08.780192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f99a4-f24a-4d78-abb2-d5473baba5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:08.780863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f99a4-f24a-4d78-abb2-d5473baba5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.781145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f99a4-f24a-4d78-abb2-d5473baba5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.781420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f99a4-f24a-4d78-abb2-d5473baba5d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.782248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f99a4-f24a-4d78-abb2-d5473baba5d4 None admin] RBAC: Adding query filter params (role.id=932e429fa28248f285663a90fc880876, user.id=8a1bfd7582fd426f9461d949b07efc50, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:08.782571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f99a4-f24a-4d78-abb2-d5473baba5d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:08.784024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f99a4-f24a-4d78-abb2-d5473baba5d4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:08.789144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 146/277] 10.222.0.32 () {56 vars in 1308 bytes} [Mon May 30 11:50:08 2022] GET /identity/v3/role_assignments?user.id=8a1bfd7582fd426f9461d949b07efc50&scope.system=all&role.id=932e429fa28248f285663a90fc880876 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:08.793683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7213fb-c77a-45e7-9d50-f89a85cd1209 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.793980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7213fb-c77a-45e7-9d50-f89a85cd1209 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.794268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7213fb-c77a-45e7-9d50-f89a85cd1209 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.796127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 147/278] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:08.797814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ae9d9-29ea-4bbf-995a-097b13709eac None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.798112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ae9d9-29ea-4bbf-995a-097b13709eac None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.798394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ae9d9-29ea-4bbf-995a-097b13709eac None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.800111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 148/279] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:08.801503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6d36-ba89-472b-972c-7a0b293b115c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.801806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6d36-ba89-472b-972c-7a0b293b115c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.802090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6d36-ba89-472b-972c-7a0b293b115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.802684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541953ec-e979-4bf2-8818-4995b720a7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:08.803795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541953ec-e979-4bf2-8818-4995b720a7fe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.804258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541953ec-e979-4bf2-8818-4995b720a7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.804677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541953ec-e979-4bf2-8818-4995b720a7fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.818146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541953ec-e979-4bf2-8818-4995b720a7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:08.821030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-664c6d36-ba89-472b-972c-7a0b293b115c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:08.823017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-664c6d36-ba89-472b-972c-7a0b293b115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-541953ec-e979-4bf2-8818-4995b720a7fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:08.827060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:08.828240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:08.829044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:08.832375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 132/280] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:08.833228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:08.834739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54e5cd-68ed-460f-b600-f858d1a2fdad None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:08.835189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54e5cd-68ed-460f-b600-f858d1a2fdad None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:08.835399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54e5cd-68ed-460f-b600-f858d1a2fdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:08.854310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2b54e5cd-68ed-460f-b600-f858d1a2fdad None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:08.855544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2b54e5cd-68ed-460f-b600-f858d1a2fdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:08.869581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:08.912821 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 149/281] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:08.913599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c6d36-ba89-472b-972c-7a0b293b115c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:08.950527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 133/282] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:08.961256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54e5cd-68ed-460f-b600-f858d1a2fdad None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.026921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c6d36-ba89-472b-972c-7a0b293b115c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.063466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.064141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.064451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.064846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.065595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:09.065907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:09.067295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:09.077266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 150/283] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:09.078010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.081756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:09.081895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54e5cd-68ed-460f-b600-f858d1a2fdad None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.122555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d196ed-f1c6-42a4-a7db-18fd161fd3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.123263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d196ed-f1c6-42a4-a7db-18fd161fd3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.123567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d196ed-f1c6-42a4-a7db-18fd161fd3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.123809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d196ed-f1c6-42a4-a7db-18fd161fd3c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.125070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d196ed-f1c6-42a4-a7db-18fd161fd3c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:09.126524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d196ed-f1c6-42a4-a7db-18fd161fd3c2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:09.128459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 134/284] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:09.133820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d196ed-f1c6-42a4-a7db-18fd161fd3c2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.159526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:09.209999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5986d-bb23-4949-9086-187e0dead163 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.252465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599f085-cf14-4030-bff3-3273640764ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.253212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f085-cf14-4030-bff3-3273640764ef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.253504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f085-cf14-4030-bff3-3273640764ef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.253790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599f085-cf14-4030-bff3-3273640764ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.260291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d196ed-f1c6-42a4-a7db-18fd161fd3c2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.265253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c599f085-cf14-4030-bff3-3273640764ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c599f085-cf14-4030-bff3-3273640764ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:09.273503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:09.275171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 151/285] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:09.276237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599f085-cf14-4030-bff3-3273640764ef None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.307388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19215627-8716-4413-829b-652dcdc9bc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.308531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215627-8716-4413-829b-652dcdc9bc67 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.309051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215627-8716-4413-829b-652dcdc9bc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.309342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19215627-8716-4413-829b-652dcdc9bc67 None admin] PATH_INFO: `/v3/users/49d5a4807e5045208cb5c45d64973f41` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.312448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19215627-8716-4413-829b-652dcdc9bc67 None admin] RBAC: Authorizing `identity:get_user(user_id=49d5a4807e5045208cb5c45d64973f41)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:09.314386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19215627-8716-4413-829b-652dcdc9bc67 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:09.321750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 135/286] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:09 2022] GET /identity/v3/users/49d5a4807e5045208cb5c45d64973f41 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:09.321750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19215627-8716-4413-829b-652dcdc9bc67 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.359190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:09.401879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599f085-cf14-4030-bff3-3273640764ef None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.414580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:09.436250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19215627-8716-4413-829b-652dcdc9bc67 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.439236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb33c4d-ad02-4e7d-99c7-c76f589b2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.439914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb33c4d-ad02-4e7d-99c7-c76f589b2b0d None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.440228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb33c4d-ad02-4e7d-99c7-c76f589b2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.440515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb33c4d-ad02-4e7d-99c7-c76f589b2b0d None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.441228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb33c4d-ad02-4e7d-99c7-c76f589b2b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:09.442692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb33c4d-ad02-4e7d-99c7-c76f589b2b0d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:09.466580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 152/287] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:50:09 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:09.473363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb33c4d-ad02-4e7d-99c7-c76f589b2b0d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.476589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.477444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.478411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.478843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.479733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:09.480147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:09.481919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:09.498452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 136/288] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:09.499594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.637945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb33c4d-ad02-4e7d-99c7-c76f589b2b0d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.643946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beb16f-10f3-4b82-bc43-3701e622ac96 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.668350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:09.688555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881726db-6a9a-43a7-a1c2-1bea1315e18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.689956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881726db-6a9a-43a7-a1c2-1bea1315e18e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.689956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881726db-6a9a-43a7-a1c2-1bea1315e18e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.690197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881726db-6a9a-43a7-a1c2-1bea1315e18e None admin] PATH_INFO: `/v3/users/c2893c458cea4aaba7e0b0969aa0040a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.691795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881726db-6a9a-43a7-a1c2-1bea1315e18e None admin] RBAC: Authorizing `identity:get_user(user_id=c2893c458cea4aaba7e0b0969aa0040a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:09.692106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6641d6-ac80-481b-a0c3-70690200643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.692796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6641d6-ac80-481b-a0c3-70690200643c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.693088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6641d6-ac80-481b-a0c3-70690200643c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.693369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6641d6-ac80-481b-a0c3-70690200643c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.693461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881726db-6a9a-43a7-a1c2-1bea1315e18e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:09.695563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 137/289] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:09 2022] GET /identity/v3/users/c2893c458cea4aaba7e0b0969aa0040a => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:09.702350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881726db-6a9a-43a7-a1c2-1bea1315e18e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.722120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:09.784311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6641d6-ac80-481b-a0c3-70690200643c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:09.785807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6641d6-ac80-481b-a0c3-70690200643c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:09.839312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881726db-6a9a-43a7-a1c2-1bea1315e18e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:09.886684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bafbfb-bafe-47b3-bf2a-b4db5a6e876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:09.889090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bafbfb-bafe-47b3-bf2a-b4db5a6e876f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:09.889090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bafbfb-bafe-47b3-bf2a-b4db5a6e876f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:09.889090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bafbfb-bafe-47b3-bf2a-b4db5a6e876f None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:09.890270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bafbfb-bafe-47b3-bf2a-b4db5a6e876f None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:09.891951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bafbfb-bafe-47b3-bf2a-b4db5a6e876f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:09.894358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 138/290] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:09 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:09.901149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bafbfb-bafe-47b3-bf2a-b4db5a6e876f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.940018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 153/291] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:09.948674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6641d6-ac80-481b-a0c3-70690200643c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:09.971626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:10.024662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:10.040011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bafbfb-bafe-47b3-bf2a-b4db5a6e876f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:10.077388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce676048-4c34-4044-aa58-d43152a3f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:10.078647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce676048-4c34-4044-aa58-d43152a3f5bd None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:10.079081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce676048-4c34-4044-aa58-d43152a3f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:10.079507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce676048-4c34-4044-aa58-d43152a3f5bd None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d/users/c2893c458cea4aaba7e0b0969aa0040a/roles/c2921011b0a949eba8e918eb306cf45b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:10.081775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce676048-4c34-4044-aa58-d43152a3f5bd None admin] RBAC: Authorizing `identity:create_grant(project_id=5229c8d118a3441fa4e48635e621fb8d, user_id=c2893c458cea4aaba7e0b0969aa0040a, role_id=c2921011b0a949eba8e918eb306cf45b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:10.083405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce676048-4c34-4044-aa58-d43152a3f5bd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:10.099805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 139/292] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:09 2022] PUT /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d/users/c2893c458cea4aaba7e0b0969aa0040a/roles/c2921011b0a949eba8e918eb306cf45b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:10.158411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6641d6-ac80-481b-a0c3-70690200643c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:10.223065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1138d-1fb1-4f03-823f-5b3087f41371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:10.224223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1138d-1fb1-4f03-823f-5b3087f41371 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:10.224672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1138d-1fb1-4f03-823f-5b3087f41371 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:10.225096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1138d-1fb1-4f03-823f-5b3087f41371 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:10.226474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1138d-1fb1-4f03-823f-5b3087f41371 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:10.228151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1138d-1fb1-4f03-823f-5b3087f41371 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:10.233160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 154/293] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:09 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:10.237683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce676048-4c34-4044-aa58-d43152a3f5bd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:10.367401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce676048-4c34-4044-aa58-d43152a3f5bd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:10.410957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f45538-b79f-47c6-8ed6-c16fffa7bb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:10.411907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f45538-b79f-47c6-8ed6-c16fffa7bb60 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:10.412367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f45538-b79f-47c6-8ed6-c16fffa7bb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:10.412796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f45538-b79f-47c6-8ed6-c16fffa7bb60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:10.413787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f45538-b79f-47c6-8ed6-c16fffa7bb60 None admin] RBAC: Adding query filter params (role.id=932e429fa28248f285663a90fc880876, scope.project.id=5229c8d118a3441fa4e48635e621fb8d, user.id=49d5a4807e5045208cb5c45d64973f41) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:10.414253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f45538-b79f-47c6-8ed6-c16fffa7bb60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:10.415912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f45538-b79f-47c6-8ed6-c16fffa7bb60 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:10.428311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 140/294] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:10 2022] GET /identity/v3/role_assignments?user.id=49d5a4807e5045208cb5c45d64973f41&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=932e429fa28248f285663a90fc880876 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:11.190923 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca72918-0af3-405a-841f-06b1641d15d3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:11.191434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca72918-0af3-405a-841f-06b1641d15d3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:11.191550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca72918-0af3-405a-841f-06b1641d15d3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:11.193151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 155/295] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:11.199694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c50a-51fe-4844-a2f1-55c45d68babd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:11.200132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c50a-51fe-4844-a2f1-55c45d68babd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:11.200355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c50a-51fe-4844-a2f1-55c45d68babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:11.206574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:11.219898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e0a6c50a-51fe-4844-a2f1-55c45d68babd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:11.221201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e0a6c50a-51fe-4844-a2f1-55c45d68babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:11.300273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 141/296] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:11.318554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca72918-0af3-405a-841f-06b1641d15d3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:11.413704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:11.428363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca72918-0af3-405a-841f-06b1641d15d3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:11.464010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf6df9-e209-427f-b323-f94a5fbc3ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:11.464781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf6df9-e209-427f-b323-f94a5fbc3ffa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:11.465071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf6df9-e209-427f-b323-f94a5fbc3ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:11.465353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf6df9-e209-427f-b323-f94a5fbc3ffa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:11.475861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf6df9-e209-427f-b323-f94a5fbc3ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-12cf6df9-e209-427f-b323-f94a5fbc3ffa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:11.486955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:11.487815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:11.488627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 156/297] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:11.494622 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6c50a-51fe-4844-a2f1-55c45d68babd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:11.613460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6c50a-51fe-4844-a2f1-55c45d68babd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:11.653332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:11.654021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:11.654310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:11.654594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:11.655357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:11.655667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:11.657110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:11.658556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:11.669026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 142/298] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:11.674429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf6df9-e209-427f-b323-f94a5fbc3ffa None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:11.784638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf6df9-e209-427f-b323-f94a5fbc3ffa None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:11.821261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ddadb-dacc-4f84-8e2d-549b45ab56a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:11.822869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddadb-dacc-4f84-8e2d-549b45ab56a8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:11.823351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddadb-dacc-4f84-8e2d-549b45ab56a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:11.823843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ddadb-dacc-4f84-8e2d-549b45ab56a8 None admin] PATH_INFO: `/v3/users/8a1bfd7582fd426f9461d949b07efc50` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:11.825315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ddadb-dacc-4f84-8e2d-549b45ab56a8 None admin] RBAC: Authorizing `identity:get_user(user_id=8a1bfd7582fd426f9461d949b07efc50)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:11.826991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ddadb-dacc-4f84-8e2d-549b45ab56a8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:11.828876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 157/299] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:11 2022] GET /identity/v3/users/8a1bfd7582fd426f9461d949b07efc50 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:11.838034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:11.965003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ed9a6-fc47-4511-951b-c9ba94682d65 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:12.001987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f71830-0e6e-4319-8a0d-a71e6769cb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:12.003225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f71830-0e6e-4319-8a0d-a71e6769cb4a None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.003710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f71830-0e6e-4319-8a0d-a71e6769cb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.004201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f71830-0e6e-4319-8a0d-a71e6769cb4a None admin] PATH_INFO: `/v3/system/users/8a1bfd7582fd426f9461d949b07efc50/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.005923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f71830-0e6e-4319-8a0d-a71e6769cb4a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8a1bfd7582fd426f9461d949b07efc50, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:12.007502 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f71830-0e6e-4319-8a0d-a71e6769cb4a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:12.020393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 143/300] 10.222.0.32 () {58 vars in 1334 bytes} [Mon May 30 11:50:11 2022] PUT /identity/v3/system/users/8a1bfd7582fd426f9461d949b07efc50/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:12.291063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35069b-9dc9-48d6-a077-3f4e44b17214 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.291678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35069b-9dc9-48d6-a077-3f4e44b17214 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.291678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35069b-9dc9-48d6-a077-3f4e44b17214 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.293067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 158/301] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:12.298555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd47d3-f646-4b7e-9803-a089accaa898 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.298729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd588f28-260d-4d9a-8dac-c844a3bf7751 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.299147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd588f28-260d-4d9a-8dac-c844a3bf7751 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.299490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd588f28-260d-4d9a-8dac-c844a3bf7751 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.299763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd47d3-f646-4b7e-9803-a089accaa898 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.300218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd47d3-f646-4b7e-9803-a089accaa898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.301190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 144/302] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:12.307622 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb585f-e4d7-43d6-acd0-63b7631dd3d4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.307915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb585f-e4d7-43d6-acd0-63b7631dd3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.308241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb585f-e4d7-43d6-acd0-63b7631dd3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.318785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-44cd47d3-f646-4b7e-9803-a089accaa898 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:12.320333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-44cd47d3-f646-4b7e-9803-a089accaa898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:12.326478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-84fb585f-e4d7-43d6-acd0-63b7631dd3d4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:12.327652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-84fb585f-e4d7-43d6-acd0-63b7631dd3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:12.333778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:12.393489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:12.406873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 159/303] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:12.412216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 145/304] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:12.430684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd47d3-f646-4b7e-9803-a089accaa898 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:12.433637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb585f-e4d7-43d6-acd0-63b7631dd3d4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:12.560624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd47d3-f646-4b7e-9803-a089accaa898 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:12.567179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb585f-e4d7-43d6-acd0-63b7631dd3d4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:12.603820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ad7f-96d6-40ea-a072-1534234f5460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:12.605238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ad7f-96d6-40ea-a072-1534234f5460 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.605521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ad7f-96d6-40ea-a072-1534234f5460 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.605798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ad7f-96d6-40ea-a072-1534234f5460 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.607975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0ad7f-96d6-40ea-a072-1534234f5460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:12.609856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0ad7f-96d6-40ea-a072-1534234f5460 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:12.616206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 160/305] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:12.619559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ad7f-96d6-40ea-a072-1534234f5460 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:12.621074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361104-7cbc-4745-9353-13b9bd0701e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:12.621980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361104-7cbc-4745-9353-13b9bd0701e9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.622339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361104-7cbc-4745-9353-13b9bd0701e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.622678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361104-7cbc-4745-9353-13b9bd0701e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.638217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05361104-7cbc-4745-9353-13b9bd0701e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:12.641250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-05361104-7cbc-4745-9353-13b9bd0701e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:12.646868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:12.647754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:12.648569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:12.652519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 146/306] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:12.654279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361104-7cbc-4745-9353-13b9bd0701e9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:12.725103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:12.749350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ad7f-96d6-40ea-a072-1534234f5460 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:12.786785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361104-7cbc-4745-9353-13b9bd0701e9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:12.796447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13c113-86fc-4ded-9484-5617eef5274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:12.797557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13c113-86fc-4ded-9484-5617eef5274c None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.798036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13c113-86fc-4ded-9484-5617eef5274c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.798553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13c113-86fc-4ded-9484-5617eef5274c None admin] PATH_INFO: `/v3/groups` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.799456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13c113-86fc-4ded-9484-5617eef5274c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:12.801149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13c113-86fc-4ded-9484-5617eef5274c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:12.822412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 161/307] 10.222.0.32 () {60 vars in 1134 bytes} [Mon May 30 11:50:12 2022] POST /identity/v3/groups => generated 229 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:12.834044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864265c-d990-4e3a-84cf-e00916b2ad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:12.835141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864265c-d990-4e3a-84cf-e00916b2ad08 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.835630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864265c-d990-4e3a-84cf-e00916b2ad08 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.836102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864265c-d990-4e3a-84cf-e00916b2ad08 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.837847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864265c-d990-4e3a-84cf-e00916b2ad08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:12.838802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864265c-d990-4e3a-84cf-e00916b2ad08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:12.840615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864265c-d990-4e3a-84cf-e00916b2ad08 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:12.851843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 147/308] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:12.857535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13c113-86fc-4ded-9484-5617eef5274c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:12.962166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:12.964618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dee09-b27e-4189-8773-7ec63c320996 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.965151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dee09-b27e-4189-8773-7ec63c320996 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.965634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dee09-b27e-4189-8773-7ec63c320996 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.967682 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 148/309] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:12.973615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cdf72-62fb-4c0f-9a0c-f6c305e2e152 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:12.974231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cdf72-62fb-4c0f-9a0c-f6c305e2e152 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:12.974981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cdf72-62fb-4c0f-9a0c-f6c305e2e152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:12.995224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c53cdf72-62fb-4c0f-9a0c-f6c305e2e152 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:12.995996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13c113-86fc-4ded-9484-5617eef5274c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:12.997074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c53cdf72-62fb-4c0f-9a0c-f6c305e2e152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:13.045699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170488-77bc-4421-a06b-9d5021ef9017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:13.047128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170488-77bc-4421-a06b-9d5021ef9017 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:13.047438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170488-77bc-4421-a06b-9d5021ef9017 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:13.047719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170488-77bc-4421-a06b-9d5021ef9017 None admin] PATH_INFO: `/v3/users/11afa163e1f04ccfb77a594112856562` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:13.065828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170488-77bc-4421-a06b-9d5021ef9017 None admin] RBAC: Authorizing `identity:get_user(user_id=11afa163e1f04ccfb77a594112856562)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:13.067317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170488-77bc-4421-a06b-9d5021ef9017 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:13.069254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 162/310] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:12 2022] GET /identity/v3/users/11afa163e1f04ccfb77a594112856562 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:13.075742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170488-77bc-4421-a06b-9d5021ef9017 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:13.088227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 149/311] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:13.108861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cdf72-62fb-4c0f-9a0c-f6c305e2e152 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:13.163391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:13.206211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170488-77bc-4421-a06b-9d5021ef9017 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:13.244384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cdf72-62fb-4c0f-9a0c-f6c305e2e152 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:13.251677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:13.259603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:13.260444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:13.260736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:13.261019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:13.262151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] RBAC: Adding query filter params (role.id=c2921011b0a949eba8e918eb306cf45b, user.id=11afa163e1f04ccfb77a594112856562, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:13.262968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:13.264988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:13.280182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 163/312] 10.222.0.32 () {56 vars in 1308 bytes} [Mon May 30 11:50:13 2022] GET /identity/v3/role_assignments?user.id=11afa163e1f04ccfb77a594112856562&scope.system=all&role.id=c2921011b0a949eba8e918eb306cf45b => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:13.293290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db266fcb-e417-4fa6-9971-864ebe78974c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:13.294124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db266fcb-e417-4fa6-9971-864ebe78974c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:13.294417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db266fcb-e417-4fa6-9971-864ebe78974c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:13.294699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db266fcb-e417-4fa6-9971-864ebe78974c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:13.307794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db266fcb-e417-4fa6-9971-864ebe78974c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-db266fcb-e417-4fa6-9971-864ebe78974c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:13.316050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:13.317031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:13.317814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:13.322708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 150/313] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:13.324257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:13.440968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016c974-1fb9-416d-83ed-e602de5afd73 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:13.482384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:13.483076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:13.483366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:13.483653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:13.484450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:13.484765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:13.486195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:13.496416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 164/314] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:13.501916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db266fcb-e417-4fa6-9971-864ebe78974c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:13.617642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db266fcb-e417-4fa6-9971-864ebe78974c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:13.619424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:13.657559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc072c-6117-49e1-a8fa-97a89498e0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:13.658440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc072c-6117-49e1-a8fa-97a89498e0fe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:13.658727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc072c-6117-49e1-a8fa-97a89498e0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:13.659007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc072c-6117-49e1-a8fa-97a89498e0fe None admin] PATH_INFO: `/v3/users/c2893c458cea4aaba7e0b0969aa0040a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:13.660276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc072c-6117-49e1-a8fa-97a89498e0fe None admin] RBAC: Authorizing `identity:get_user(user_id=c2893c458cea4aaba7e0b0969aa0040a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:13.661748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc072c-6117-49e1-a8fa-97a89498e0fe None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:13.668272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 151/315] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:13 2022] GET /identity/v3/users/c2893c458cea4aaba7e0b0969aa0040a => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:13.670148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:13.733208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:13.784034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25314df-badb-49e7-855e-ae4c04be9813 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:13.819808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2be54f-4a52-4be4-bb57-85f4dd73a606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:13.820491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be54f-4a52-4be4-bb57-85f4dd73a606 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:13.820782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be54f-4a52-4be4-bb57-85f4dd73a606 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:13.821064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be54f-4a52-4be4-bb57-85f4dd73a606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:13.898176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2be54f-4a52-4be4-bb57-85f4dd73a606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:13.900220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2be54f-4a52-4be4-bb57-85f4dd73a606 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:13.970537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:14.023493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 165/316] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:14.025455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc072c-6117-49e1-a8fa-97a89498e0fe None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:14.141500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc072c-6117-49e1-a8fa-97a89498e0fe None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:14.178437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd64a9-a117-4201-9b73-7396c8a042d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:14.179322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd64a9-a117-4201-9b73-7396c8a042d8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:14.179612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd64a9-a117-4201-9b73-7396c8a042d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:14.179898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd64a9-a117-4201-9b73-7396c8a042d8 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:14.181431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd64a9-a117-4201-9b73-7396c8a042d8 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:14.182935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd64a9-a117-4201-9b73-7396c8a042d8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:14.189531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 152/317] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:14 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:14.190920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2be54f-4a52-4be4-bb57-85f4dd73a606 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:14.300865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2be54f-4a52-4be4-bb57-85f4dd73a606 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:14.337551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7d916-18b5-4e61-93d6-19880fc19343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:14.338479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7d916-18b5-4e61-93d6-19880fc19343 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:14.338479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7d916-18b5-4e61-93d6-19880fc19343 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:14.338800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7d916-18b5-4e61-93d6-19880fc19343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:14.339574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7d916-18b5-4e61-93d6-19880fc19343 None admin] RBAC: Adding query filter params (role.id=c2921011b0a949eba8e918eb306cf45b, scope.project.id=5229c8d118a3441fa4e48635e621fb8d, user.id=c2893c458cea4aaba7e0b0969aa0040a) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:14.339890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7d916-18b5-4e61-93d6-19880fc19343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:14.341371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7d916-18b5-4e61-93d6-19880fc19343 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:14.352273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 166/318] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:14 2022] GET /identity/v3/role_assignments?user.id=c2893c458cea4aaba7e0b0969aa0040a&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=c2921011b0a949eba8e918eb306cf45b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:14.369983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927c64d-b668-4f4e-9263-b64e214bf272 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:14.370631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927c64d-b668-4f4e-9263-b64e214bf272 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:14.370631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927c64d-b668-4f4e-9263-b64e214bf272 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:14.372396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 153/319] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:14.377762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3748cc7-9be8-48e5-a4f9-4e7500a5843e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:14.378054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3748cc7-9be8-48e5-a4f9-4e7500a5843e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:14.378338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3748cc7-9be8-48e5-a4f9-4e7500a5843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:14.397474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c3748cc7-9be8-48e5-a4f9-4e7500a5843e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:14.398733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c3748cc7-9be8-48e5-a4f9-4e7500a5843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:14.419804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:14.480008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 167/320] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:14.503344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927c64d-b668-4f4e-9263-b64e214bf272 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:14.556838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:14.625382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927c64d-b668-4f4e-9263-b64e214bf272 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:14.662617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1967b65-a954-4975-9fb6-546da6a95d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:14.663434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1967b65-a954-4975-9fb6-546da6a95d51 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:14.663719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1967b65-a954-4975-9fb6-546da6a95d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:14.663995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1967b65-a954-4975-9fb6-546da6a95d51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:14.675227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1967b65-a954-4975-9fb6-546da6a95d51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a1967b65-a954-4975-9fb6-546da6a95d51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:14.683617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:14.684592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:14.685372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 154/321] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:14.692041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3748cc7-9be8-48e5-a4f9-4e7500a5843e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:14.796111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3748cc7-9be8-48e5-a4f9-4e7500a5843e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:14.831615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:14.832246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:14.832537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:14.832816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:14.833566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:14.833877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:14.835276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:14.844794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 168/322] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:14.850048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1967b65-a954-4975-9fb6-546da6a95d51 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:14.954497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:14.960015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1967b65-a954-4975-9fb6-546da6a95d51 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:14.996795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf69a37-a6eb-495f-b974-0b9f0e595edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:14.997681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf69a37-a6eb-495f-b974-0b9f0e595edb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:14.997967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf69a37-a6eb-495f-b974-0b9f0e595edb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:14.998244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf69a37-a6eb-495f-b974-0b9f0e595edb None admin] PATH_INFO: `/v3/users/8a1bfd7582fd426f9461d949b07efc50` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:14.999475 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf69a37-a6eb-495f-b974-0b9f0e595edb None admin] RBAC: Authorizing `identity:get_user(user_id=8a1bfd7582fd426f9461d949b07efc50)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:15.000945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf69a37-a6eb-495f-b974-0b9f0e595edb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:15.002914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 155/323] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:14 2022] GET /identity/v3/users/8a1bfd7582fd426f9461d949b07efc50 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:15.008380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:15.026099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:15.120053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65b86b-9b39-4c8b-8dca-5bd7bc5cd749 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:15.154912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19bb50-4b19-4533-89e3-1f53ae86249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:15.155579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19bb50-4b19-4533-89e3-1f53ae86249a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:15.155867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19bb50-4b19-4533-89e3-1f53ae86249a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:15.156148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19bb50-4b19-4533-89e3-1f53ae86249a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:15.157009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19bb50-4b19-4533-89e3-1f53ae86249a None admin] RBAC: Adding query filter params (role.id=932e429fa28248f285663a90fc880876, user.id=8a1bfd7582fd426f9461d949b07efc50, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:15.157327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19bb50-4b19-4533-89e3-1f53ae86249a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:15.158777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19bb50-4b19-4533-89e3-1f53ae86249a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:15.171806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 169/324] 10.222.0.32 () {56 vars in 1308 bytes} [Mon May 30 11:50:15 2022] GET /identity/v3/role_assignments?user.id=8a1bfd7582fd426f9461d949b07efc50&scope.system=all&role.id=932e429fa28248f285663a90fc880876 => generated 510 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:15.295030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63acec-9d9c-4d92-9131-d99377227428 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:15.295382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63acec-9d9c-4d92-9131-d99377227428 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:15.295598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63acec-9d9c-4d92-9131-d99377227428 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:15.297185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 156/325] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:15.302659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4428-9e71-4f3e-a199-e54a3d5f4acb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:15.303008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4428-9e71-4f3e-a199-e54a3d5f4acb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:15.303227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4428-9e71-4f3e-a199-e54a3d5f4acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:15.323256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7fed4428-9e71-4f3e-a199-e54a3d5f4acb None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:15.324543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7fed4428-9e71-4f3e-a199-e54a3d5f4acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:15.414168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 170/326] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:15.438199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63acec-9d9c-4d92-9131-d99377227428 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:15.539907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63acec-9d9c-4d92-9131-d99377227428 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:15.573583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debcc6f-c497-4a33-9668-d2a4aba45dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:15.574388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcc6f-c497-4a33-9668-d2a4aba45dee None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:15.574674 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcc6f-c497-4a33-9668-d2a4aba45dee None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:15.574954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcc6f-c497-4a33-9668-d2a4aba45dee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:15.576126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debcc6f-c497-4a33-9668-d2a4aba45dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:15.577594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debcc6f-c497-4a33-9668-d2a4aba45dee None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:15.579375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 157/327] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:15.584947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4428-9e71-4f3e-a199-e54a3d5f4acb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:15.587550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:15.684350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fb80f-2ed9-4195-ad69-775192bf1767 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:15.684568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fb80f-2ed9-4195-ad69-775192bf1767 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:15.684843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fb80f-2ed9-4195-ad69-775192bf1767 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:15.686221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 158/328] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:50:15.688454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4428-9e71-4f3e-a199-e54a3d5f4acb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:15.691276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9537e-ee0d-4d76-a045-af90f9ca6374 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:15.691568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9537e-ee0d-4d76-a045-af90f9ca6374 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:15.691850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9537e-ee0d-4d76-a045-af90f9ca6374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:15.708735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dfc9537e-ee0d-4d76-a045-af90f9ca6374 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:15.709932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dfc9537e-ee0d-4d76-a045-af90f9ca6374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:15.723952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3b65b-d90e-4947-88eb-0ff6e83a8de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:15.724658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3b65b-d90e-4947-88eb-0ff6e83a8de2 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:15.724938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3b65b-d90e-4947-88eb-0ff6e83a8de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:15.725213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3b65b-d90e-4947-88eb-0ff6e83a8de2 None admin] PATH_INFO: `/v3/groups` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:15.725892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3b65b-d90e-4947-88eb-0ff6e83a8de2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:15.727343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3b65b-d90e-4947-88eb-0ff6e83a8de2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:15.737922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:15.744794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 171/329] 10.222.0.32 () {60 vars in 1134 bytes} [Mon May 30 11:50:15 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:15.784373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 159/330] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:15.809171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3b65b-d90e-4947-88eb-0ff6e83a8de2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:15.875352 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:15.919088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3b65b-d90e-4947-88eb-0ff6e83a8de2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:15.956627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b768148-e941-42a7-8689-01d154b72c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:15.957408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b768148-e941-42a7-8689-01d154b72c21 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:15.957689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b768148-e941-42a7-8689-01d154b72c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:15.957962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b768148-e941-42a7-8689-01d154b72c21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:15.968966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b768148-e941-42a7-8689-01d154b72c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1b768148-e941-42a7-8689-01d154b72c21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:15.976811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:15.978435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:15.979514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 172/331] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:15.984561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9537e-ee0d-4d76-a045-af90f9ca6374 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:16.098509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9537e-ee0d-4d76-a045-af90f9ca6374 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:16.137780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:16.138854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:16.139137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:16.139412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:16.140191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:16.140532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:16.141921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:16.152382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 160/332] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:16.157959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b768148-e941-42a7-8689-01d154b72c21 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:16.272525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b768148-e941-42a7-8689-01d154b72c21 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:16.304847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:16.318213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60412b-d849-4df3-932b-14dc375b93b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:16.319112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60412b-d849-4df3-932b-14dc375b93b2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:16.319399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60412b-d849-4df3-932b-14dc375b93b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:16.319679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60412b-d849-4df3-932b-14dc375b93b2 None admin] PATH_INFO: `/v3/users/11afa163e1f04ccfb77a594112856562` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:16.320926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60412b-d849-4df3-932b-14dc375b93b2 None admin] RBAC: Authorizing `identity:get_user(user_id=11afa163e1f04ccfb77a594112856562)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:16.323005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60412b-d849-4df3-932b-14dc375b93b2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:16.325134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 173/333] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:16 2022] GET /identity/v3/users/11afa163e1f04ccfb77a594112856562 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:16.330541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:16.373799 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:16.456676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ace4e6-1cde-48cc-8a83-ec27b4f42b18 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:16.494396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c4cbf-6153-463c-a833-64ad41de1853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:16.495270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c4cbf-6153-463c-a833-64ad41de1853 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:16.495552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c4cbf-6153-463c-a833-64ad41de1853 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:16.495830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c4cbf-6153-463c-a833-64ad41de1853 None admin] PATH_INFO: `/v3/system/users/11afa163e1f04ccfb77a594112856562/roles/c2921011b0a949eba8e918eb306cf45b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:16.497360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c4cbf-6153-463c-a833-64ad41de1853 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=11afa163e1f04ccfb77a594112856562, role_id=c2921011b0a949eba8e918eb306cf45b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:16.498817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c4cbf-6153-463c-a833-64ad41de1853 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:16.510300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 161/334] 10.222.0.32 () {58 vars in 1334 bytes} [Mon May 30 11:50:16 2022] PUT /identity/v3/system/users/11afa163e1f04ccfb77a594112856562/roles/c2921011b0a949eba8e918eb306cf45b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:18.703489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0dadc-75e2-4394-83b8-44c9ceb3a4c4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:18.704503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0dadc-75e2-4394-83b8-44c9ceb3a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:18.704999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0dadc-75e2-4394-83b8-44c9ceb3a4c4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:18.706809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 174/335] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:18.713143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c5f68-d61c-4c49-9330-075048ceb107 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:18.713633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c5f68-d61c-4c49-9330-075048ceb107 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:18.713708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c5f68-d61c-4c49-9330-075048ceb107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:18.733655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6d5c5f68-d61c-4c49-9330-075048ceb107 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:18.734854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6d5c5f68-d61c-4c49-9330-075048ceb107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:18.816945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 162/336] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:18.854486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0dadc-75e2-4394-83b8-44c9ceb3a4c4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:18.928232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5108-569e-42f0-90a0-4d31f1a07ad4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:18.928926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5108-569e-42f0-90a0-4d31f1a07ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:18.929445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5108-569e-42f0-90a0-4d31f1a07ad4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:18.931089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 163/337] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:18.941687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed6e2b-04cd-4b67-b144-ad1d0dc9d17a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:18.942285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed6e2b-04cd-4b67-b144-ad1d0dc9d17a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:18.942796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed6e2b-04cd-4b67-b144-ad1d0dc9d17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:18.980030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f6ed6e2b-04cd-4b67-b144-ad1d0dc9d17a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:18.981588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f6ed6e2b-04cd-4b67-b144-ad1d0dc9d17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:18.989873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0dadc-75e2-4394-83b8-44c9ceb3a4c4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:19.011865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:19.035948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ee13a-3aa5-4a4a-bf99-af2c5b55935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:19.036813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ee13a-3aa5-4a4a-bf99-af2c5b55935d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:19.037096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ee13a-3aa5-4a4a-bf99-af2c5b55935d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:19.037376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ee13a-3aa5-4a4a-bf99-af2c5b55935d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:19.063505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ee13a-3aa5-4a4a-bf99-af2c5b55935d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e78ee13a-3aa5-4a4a-bf99-af2c5b55935d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:19.071916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:19.072999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:19.073789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:19.076508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 175/338] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:19.080327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6f7c-d917-4be2-b40e-a33bed638aa8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:19.080891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6f7c-d917-4be2-b40e-a33bed638aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:19.081390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6f7c-d917-4be2-b40e-a33bed638aa8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:19.083717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 176/339] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:19.084823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f6f7c-d917-4be2-b40e-a33bed638aa8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:19.135612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 164/340] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:19.136897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c1854-143b-44c6-9c5b-73626aea0b55 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:19.137193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c1854-143b-44c6-9c5b-73626aea0b55 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:19.137473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c1854-143b-44c6-9c5b-73626aea0b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:19.164625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:19.166637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fb8c1854-143b-44c6-9c5b-73626aea0b55 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:19.167859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fb8c1854-143b-44c6-9c5b-73626aea0b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:19.241039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f6f7c-d917-4be2-b40e-a33bed638aa8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:19.282561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 165/341] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:19.283233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c1854-143b-44c6-9c5b-73626aea0b55 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:19.306713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:19.307599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:19.307886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:19.308180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:19.308946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:19.309260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:19.310666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:19.339113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 177/342] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:19.347644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:19.472638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:19.507696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c1854-143b-44c6-9c5b-73626aea0b55 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:19.550832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d708b-0078-4e5b-948e-910abcd4825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:19.551673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d708b-0078-4e5b-948e-910abcd4825a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:19.551958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d708b-0078-4e5b-948e-910abcd4825a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:19.552261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d708b-0078-4e5b-948e-910abcd4825a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:19.563695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d708b-0078-4e5b-948e-910abcd4825a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ac3d708b-0078-4e5b-948e-910abcd4825a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:19.577951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:19.578883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:19.579657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:19.581756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 166/343] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:19.582701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d708b-0078-4e5b-948e-910abcd4825a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:19.592432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374b2cb-f9cf-4707-ad3a-ce6d049bbeb1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:19.620905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:19.659249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ba909-0389-4d79-94d4-57ba219ac3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:19.660338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ba909-0389-4d79-94d4-57ba219ac3ed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:19.660800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ba909-0389-4d79-94d4-57ba219ac3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:19.661216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ba909-0389-4d79-94d4-57ba219ac3ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:19.677877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ba909-0389-4d79-94d4-57ba219ac3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8a7ba909-0389-4d79-94d4-57ba219ac3ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:19.691649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:19.692630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:19.693414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:19.695563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 178/344] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:19.702159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ba909-0389-4d79-94d4-57ba219ac3ed None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:19.718545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d708b-0078-4e5b-948e-910abcd4825a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:19.780430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ce48f-5eac-465b-bb0c-6bb12b9d4925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:19.781114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ce48f-5eac-465b-bb0c-6bb12b9d4925 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:19.781401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ce48f-5eac-465b-bb0c-6bb12b9d4925 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:19.781680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ce48f-5eac-465b-bb0c-6bb12b9d4925 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:19.801186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:19.884644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ba909-0389-4d79-94d4-57ba219ac3ed None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:19.898401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ce48f-5eac-465b-bb0c-6bb12b9d4925 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:19.901470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ce48f-5eac-465b-bb0c-6bb12b9d4925 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:19.932179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:19.933513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:19.933960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:19.934560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:19.936762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:19.937527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:19.939561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:19.950877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 179/345] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:19.951968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:20.010842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:20.042236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 167/346] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:20.043997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0b762-f761-4989-8dc3-eb55a679bcbf None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.048584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0b762-f761-4989-8dc3-eb55a679bcbf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.049145 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0b762-f761-4989-8dc3-eb55a679bcbf None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.050695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 168/347] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:20.051691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0b762-f761-4989-8dc3-eb55a679bcbf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:20.106053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:20.110445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3700683-b340-468e-92a4-b2b59f5b200c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:20.154124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8384fe0-7eb2-4971-ae61-c74498fd225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:20.155046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8384fe0-7eb2-4971-ae61-c74498fd225b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.155480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8384fe0-7eb2-4971-ae61-c74498fd225b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.155920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8384fe0-7eb2-4971-ae61-c74498fd225b None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.156971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8384fe0-7eb2-4971-ae61-c74498fd225b None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:20.157545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8384fe0-7eb2-4971-ae61-c74498fd225b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:20.159143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8384fe0-7eb2-4971-ae61-c74498fd225b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:20.170543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 180/348] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:20.172212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9065b47-307d-41b0-819f-cade9fc61d94 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.172681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9065b47-307d-41b0-819f-cade9fc61d94 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.173218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9065b47-307d-41b0-819f-cade9fc61d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.178105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0b762-f761-4989-8dc3-eb55a679bcbf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:20.192801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c9065b47-307d-41b0-819f-cade9fc61d94 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:20.194336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c9065b47-307d-41b0-819f-cade9fc61d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:20.219174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e785-750f-4b1f-a7a7-756fead30e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:20.219867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e785-750f-4b1f-a7a7-756fead30e3a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.220321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e785-750f-4b1f-a7a7-756fead30e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.220620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5e785-750f-4b1f-a7a7-756fead30e3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.278988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 181/349] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:20.280267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9065b47-307d-41b0-819f-cade9fc61d94 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:20.301460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e785-750f-4b1f-a7a7-756fead30e3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:20.302866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5e785-750f-4b1f-a7a7-756fead30e3a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:20.311406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:20.350288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:20.393520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9065b47-307d-41b0-819f-cade9fc61d94 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:20.423880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 169/350] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:20.427494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e785-750f-4b1f-a7a7-756fead30e3a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:20.436207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82462f1-824a-4786-944c-b50449ed616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:20.437094 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82462f1-824a-4786-944c-b50449ed616e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.437379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82462f1-824a-4786-944c-b50449ed616e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.437660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82462f1-824a-4786-944c-b50449ed616e None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.438861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82462f1-824a-4786-944c-b50449ed616e None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:20.440583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82462f1-824a-4786-944c-b50449ed616e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:20.442054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 182/351] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:20 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:20.442877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82462f1-824a-4786-944c-b50449ed616e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:20.559634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:20.566282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5e785-750f-4b1f-a7a7-756fead30e3a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:20.567381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82462f1-824a-4786-944c-b50449ed616e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:20.607656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6853721-51e5-4164-baa9-0b0649113a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:20.608373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6853721-51e5-4164-baa9-0b0649113a93 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.608661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6853721-51e5-4164-baa9-0b0649113a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.608943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6853721-51e5-4164-baa9-0b0649113a93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.609825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114001d-5d5a-41df-b2b1-950b1e9bde65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:20.610631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114001d-5d5a-41df-b2b1-950b1e9bde65 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.610920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114001d-5d5a-41df-b2b1-950b1e9bde65 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.611201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114001d-5d5a-41df-b2b1-950b1e9bde65 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.624079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:20.626539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114001d-5d5a-41df-b2b1-950b1e9bde65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2114001d-5d5a-41df-b2b1-950b1e9bde65 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:20.636184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:20.637108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:20.637891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 183/352] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:20.639911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec2546-068e-4faa-8027-bf55851f2936 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.639911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec2546-068e-4faa-8027-bf55851f2936 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.639911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec2546-068e-4faa-8027-bf55851f2936 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.642286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 184/353] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:20.642906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec2546-068e-4faa-8027-bf55851f2936 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:20.700771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6853721-51e5-4164-baa9-0b0649113a93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:20.702324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6853721-51e5-4164-baa9-0b0649113a93 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:20.775984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec2546-068e-4faa-8027-bf55851f2936 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:20.814246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 170/354] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:20.815738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6853721-51e5-4164-baa9-0b0649113a93 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:20.817063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ef510-ca58-4927-a693-17a5870c6a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:20.817940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ef510-ca58-4927-a693-17a5870c6a4d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.818242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ef510-ca58-4927-a693-17a5870c6a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.818502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ef510-ca58-4927-a693-17a5870c6a4d None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.820110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ef510-ca58-4927-a693-17a5870c6a4d None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:20.822027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ef510-ca58-4927-a693-17a5870c6a4d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:20.825004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 185/355] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:20 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:20.826346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6da0-856b-4c4a-a06f-9ec4cec26504 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.826785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6da0-856b-4c4a-a06f-9ec4cec26504 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.827208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6da0-856b-4c4a-a06f-9ec4cec26504 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.828673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 186/356] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:20.829609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f6da0-856b-4c4a-a06f-9ec4cec26504 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:20.857515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:20.864438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:20.942897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6853721-51e5-4164-baa9-0b0649113a93 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:20.956301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f6da0-856b-4c4a-a06f-9ec4cec26504 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:20.984937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d6de8-2b8a-4c3e-9971-aba82dcd8fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:20.985809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6de8-2b8a-4c3e-9971-aba82dcd8fdf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.986098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6de8-2b8a-4c3e-9971-aba82dcd8fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.986898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6de8-2b8a-4c3e-9971-aba82dcd8fdf None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.996948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:20.997317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d6de8-2b8a-4c3e-9971-aba82dcd8fdf None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:20.997631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:20.997920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:20.998204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:20.998829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d6de8-2b8a-4c3e-9971-aba82dcd8fdf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:20.998946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:20.999259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:21.001407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:21.001516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 171/357] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:20 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:21.002457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a512-d137-4a98-813c-087409360a55 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.002770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a512-d137-4a98-813c-087409360a55 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.003035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37a512-d137-4a98-813c-087409360a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.014428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 187/358] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:21.015159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.025219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3f37a512-d137-4a98-813c-087409360a55 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:21.026425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3f37a512-d137-4a98-813c-087409360a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:21.108526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 172/359] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:21.109444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37a512-d137-4a98-813c-087409360a55 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.123826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:21.144195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e69dc-5ac0-4eef-91d1-e17e60f2a421 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:21.161483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:21.189497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672825f-965b-4db1-bfdb-96045e431f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:21.191079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672825f-965b-4db1-bfdb-96045e431f49 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.191079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672825f-965b-4db1-bfdb-96045e431f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.191079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672825f-965b-4db1-bfdb-96045e431f49 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.192285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672825f-965b-4db1-bfdb-96045e431f49 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:21.193743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672825f-965b-4db1-bfdb-96045e431f49 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:21.195467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 188/360] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:21.196908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf6bd0-1d38-41d9-b506-dbd62edb3a0a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.197395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf6bd0-1d38-41d9-b506-dbd62edb3a0a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.197678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf6bd0-1d38-41d9-b506-dbd62edb3a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.216420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1ecf6bd0-1d38-41d9-b506-dbd62edb3a0a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:21.217621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ecf6bd0-1d38-41d9-b506-dbd62edb3a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:21.250180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37a512-d137-4a98-813c-087409360a55 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:21.289247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5930b-b07f-4dcf-99f0-02be40245024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:21.290914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5930b-b07f-4dcf-99f0-02be40245024 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.291493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5930b-b07f-4dcf-99f0-02be40245024 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.291686 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5930b-b07f-4dcf-99f0-02be40245024 None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.294330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5930b-b07f-4dcf-99f0-02be40245024 None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:21.295836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5930b-b07f-4dcf-99f0-02be40245024 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:21.297848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 173/361] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:21.298616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5930b-b07f-4dcf-99f0-02be40245024 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.298721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 189/362] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:21.299353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf6bd0-1d38-41d9-b506-dbd62edb3a0a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.394306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:21.415903 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf6bd0-1d38-41d9-b506-dbd62edb3a0a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:21.429874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5930b-b07f-4dcf-99f0-02be40245024 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:21.431914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:21.465666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9315b-eb05-4330-b4c5-7b7f6f8a88a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:21.466499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9315b-eb05-4330-b4c5-7b7f6f8a88a7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.466822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9315b-eb05-4330-b4c5-7b7f6f8a88a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.467301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9315b-eb05-4330-b4c5-7b7f6f8a88a7 None admin] PATH_INFO: `/v3/users/11afa163e1f04ccfb77a594112856562` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.469406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9315b-eb05-4330-b4c5-7b7f6f8a88a7 None admin] RBAC: Authorizing `identity:get_user(user_id=11afa163e1f04ccfb77a594112856562)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:21.470957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9315b-eb05-4330-b4c5-7b7f6f8a88a7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:21.473452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 190/363] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/users/11afa163e1f04ccfb77a594112856562 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:21.474068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9315b-eb05-4330-b4c5-7b7f6f8a88a7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.483880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:21.484985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.485331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.486967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.488505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] RBAC: Adding query filter params (group.id=50e7aa331f1e49e2a0870ced5186e649, role.id=932e429fa28248f285663a90fc880876, scope.project.id=5229c8d118a3441fa4e48635e621fb8d) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:21.489049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:21.491224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:21.501984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 174/364] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/role_assignments?group.id=50e7aa331f1e49e2a0870ced5186e649&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=932e429fa28248f285663a90fc880876 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:21.503349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.632703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9315b-eb05-4330-b4c5-7b7f6f8a88a7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:21.660875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a7f81-6e7d-4ed9-99ff-7795e0d4b2bd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:21.686208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc2067-8706-4bc3-91e7-3aaa84c7942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:21.687663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc2067-8706-4bc3-91e7-3aaa84c7942a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.687948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc2067-8706-4bc3-91e7-3aaa84c7942a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.688254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc2067-8706-4bc3-91e7-3aaa84c7942a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.704866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc2067-8706-4bc3-91e7-3aaa84c7942a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b3cc2067-8706-4bc3-91e7-3aaa84c7942a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:21.714105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:21.715231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:21.716119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 191/365] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:21.716861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8c466-840f-4915-a463-0cdebd1ead5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:21.716959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc2067-8706-4bc3-91e7-3aaa84c7942a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.717038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c466-840f-4915-a463-0cdebd1ead5f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.719220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c466-840f-4915-a463-0cdebd1ead5f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.719220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c466-840f-4915-a463-0cdebd1ead5f None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.719220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8c466-840f-4915-a463-0cdebd1ead5f None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:21.720102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee8c466-840f-4915-a463-0cdebd1ead5f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:21.722351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 175/366] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:21.723154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8c466-840f-4915-a463-0cdebd1ead5f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.734008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:21.765451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:21.862237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc2067-8706-4bc3-91e7-3aaa84c7942a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:21.865263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8c466-840f-4915-a463-0cdebd1ead5f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:21.902820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:21.903483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.903767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.904046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.904951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] RBAC: Adding query filter params (group.id=50e7aa331f1e49e2a0870ced5186e649, role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:21.905267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:21.906732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:21.910210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cef72-74f3-43ea-acd3-6d57d4d75fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:21.911340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cef72-74f3-43ea-acd3-6d57d4d75fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:21.911840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cef72-74f3-43ea-acd3-6d57d4d75fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:21.912369 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cef72-74f3-43ea-acd3-6d57d4d75fb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:21.912888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 192/367] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/role_assignments?group.id=50e7aa331f1e49e2a0870ced5186e649&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:21.913531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:21.926621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cef72-74f3-43ea-acd3-6d57d4d75fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ca1cef72-74f3-43ea-acd3-6d57d4d75fb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:21.936347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:21.937297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:21.938080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:21.940746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 176/368] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:21.941846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cef72-74f3-43ea-acd3-6d57d4d75fb6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:22.004217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:22.036961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c54a69-3c5a-4fe0-8bb9-8ef28574b7b1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:22.066325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:22.074663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cef72-74f3-43ea-acd3-6d57d4d75fb6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:22.080566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:22.081586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:22.082058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:22.082509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:22.084293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] RBAC: Adding query filter params (role.id=c2921011b0a949eba8e918eb306cf45b, user.id=11afa163e1f04ccfb77a594112856562, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:22.084856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:22.088186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:22.106334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 193/369] 10.222.0.32 () {56 vars in 1308 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/role_assignments?user.id=11afa163e1f04ccfb77a594112856562&scope.system=all&role.id=c2921011b0a949eba8e918eb306cf45b => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:22.108416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:22.116066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:22.116918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:22.117273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:22.117624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:22.119002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:22.119377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:22.121117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:22.134807 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 177/370] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:22.135729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:22.304405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d8d5c-e026-4ddc-9719-6db4679bb442 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:22.316043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2ed80-2096-4d22-8ee0-0e6192b3e0f7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:22.365628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:22.366321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:22.366759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:22.367216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:22.368690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:22.369270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:22.371812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:22.374606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:22.382102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:22.383472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:22.384606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:22.385247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:22.386299 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] RBAC: Adding query filter params (group.id=50e7aa331f1e49e2a0870ced5186e649, role.id=932e429fa28248f285663a90fc880876, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:22.386840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:22.388328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 178/371] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:22.388778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:22.389206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:22.398684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 194/372] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:22 2022] GET /identity/v3/role_assignments?group.id=50e7aa331f1e49e2a0870ced5186e649&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=932e429fa28248f285663a90fc880876 => generated 260 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:22.399665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:22.422626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:22.619927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a98d0-0674-4b4e-b7dd-f7d005927d28 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:22.638629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d75fc-e23e-4396-be0d-efb3e30b0d26 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:22.684591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8586e16-8b3f-41c0-aa68-ebb0cb135bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:22.685254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8586e16-8b3f-41c0-aa68-ebb0cb135bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:22.685539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8586e16-8b3f-41c0-aa68-ebb0cb135bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:22.685816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8586e16-8b3f-41c0-aa68-ebb0cb135bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:22.694378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98aac2-21c1-48c3-98f8-e797db9ce025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:22.695383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98aac2-21c1-48c3-98f8-e797db9ce025 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:22.695887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98aac2-21c1-48c3-98f8-e797db9ce025 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:22.697415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98aac2-21c1-48c3-98f8-e797db9ce025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:22.780133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:22.790725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98aac2-21c1-48c3-98f8-e797db9ce025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:22.792565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98aac2-21c1-48c3-98f8-e797db9ce025 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:22.845519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8586e16-8b3f-41c0-aa68-ebb0cb135bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:22.846939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8586e16-8b3f-41c0-aa68-ebb0cb135bd5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:22.895657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:22.929463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 195/373] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:22.941331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98aac2-21c1-48c3-98f8-e797db9ce025 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:23.069880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 179/374] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:23.076611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8586e16-8b3f-41c0-aa68-ebb0cb135bd5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:23.077233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:23.085899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98aac2-21c1-48c3-98f8-e797db9ce025 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:23.127259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a72afe-a005-41be-b744-42f4806171f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:23.128562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a72afe-a005-41be-b744-42f4806171f9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:23.129053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a72afe-a005-41be-b744-42f4806171f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:23.129800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a72afe-a005-41be-b744-42f4806171f9 None admin] PATH_INFO: `/v3/projects/41eaf29f1f484186a95e357823daad72` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:23.131520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a72afe-a005-41be-b744-42f4806171f9 None admin] RBAC: Authorizing `identity:get_project(project_id=41eaf29f1f484186a95e357823daad72)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:23.133247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a72afe-a005-41be-b744-42f4806171f9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:23.137149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 196/375] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:22 2022] GET /identity/v3/projects/41eaf29f1f484186a95e357823daad72 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:23.146407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a72afe-a005-41be-b744-42f4806171f9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:23.278950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8586e16-8b3f-41c0-aa68-ebb0cb135bd5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:23.300570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:23.318393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a72afe-a005-41be-b744-42f4806171f9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:23.326227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bc798-31c2-4f55-b5e5-a846f25e7664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:23.327407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bc798-31c2-4f55-b5e5-a846f25e7664 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:23.327897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bc798-31c2-4f55-b5e5-a846f25e7664 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:23.328424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bc798-31c2-4f55-b5e5-a846f25e7664 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:23.329852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bc798-31c2-4f55-b5e5-a846f25e7664 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:23.331618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bc798-31c2-4f55-b5e5-a846f25e7664 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:23.334594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 180/376] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:23 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:23.345168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bc798-31c2-4f55-b5e5-a846f25e7664 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:23.417608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819a30e-af91-44d9-bfdb-616ce14be0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:23.418840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819a30e-af91-44d9-bfdb-616ce14be0bc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:23.419318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819a30e-af91-44d9-bfdb-616ce14be0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:23.419786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819a30e-af91-44d9-bfdb-616ce14be0bc None admin] PATH_INFO: `/v3/groups/1c182bca33b2411e8eee0e74f3b7c6e5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:23.445015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819a30e-af91-44d9-bfdb-616ce14be0bc None admin] RBAC: Authorizing `identity:get_group(group_id=1c182bca33b2411e8eee0e74f3b7c6e5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:23.448098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819a30e-af91-44d9-bfdb-616ce14be0bc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:23.450426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 197/377] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:23 2022] GET /identity/v3/groups/1c182bca33b2411e8eee0e74f3b7c6e5 => generated 229 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:23.456403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819a30e-af91-44d9-bfdb-616ce14be0bc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:23.475551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bc798-31c2-4f55-b5e5-a846f25e7664 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:23.491343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:23.526382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3bed0-7136-4231-82a3-81708775426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:23.527548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3bed0-7136-4231-82a3-81708775426c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:23.527841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3bed0-7136-4231-82a3-81708775426c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:23.528125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3bed0-7136-4231-82a3-81708775426c None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:23.529833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3bed0-7136-4231-82a3-81708775426c None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:23.531845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3bed0-7136-4231-82a3-81708775426c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:23.537912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 181/378] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:23 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:23.543159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3bed0-7136-4231-82a3-81708775426c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:23.605188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819a30e-af91-44d9-bfdb-616ce14be0bc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:23.615385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:23.675208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c990d8-509b-433a-847a-0ee33b30caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:23.675799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c990d8-509b-433a-847a-0ee33b30caed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:23.676089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c990d8-509b-433a-847a-0ee33b30caed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:23.676399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c990d8-509b-433a-847a-0ee33b30caed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:23.677236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c990d8-509b-433a-847a-0ee33b30caed None admin] RBAC: Adding query filter params (group.id=1c182bca33b2411e8eee0e74f3b7c6e5, role.id=4d21498572b14478a996ac538be5382e, scope.project.id=41eaf29f1f484186a95e357823daad72) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:23.677556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c990d8-509b-433a-847a-0ee33b30caed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:23.679002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c990d8-509b-433a-847a-0ee33b30caed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:23.684604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3bed0-7136-4231-82a3-81708775426c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:23.689815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 198/379] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:23 2022] GET /identity/v3/role_assignments?group.id=1c182bca33b2411e8eee0e74f3b7c6e5&scope.project.id=41eaf29f1f484186a95e357823daad72&role.id=4d21498572b14478a996ac538be5382e => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:23.727245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8f26d-ec2b-4de7-9deb-90485616968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:23.728201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f26d-ec2b-4de7-9deb-90485616968f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:23.728490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f26d-ec2b-4de7-9deb-90485616968f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:23.728775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f26d-ec2b-4de7-9deb-90485616968f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:23.729642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8f26d-ec2b-4de7-9deb-90485616968f None admin] RBAC: Adding query filter params (group.id=50e7aa331f1e49e2a0870ced5186e649, role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=5229c8d118a3441fa4e48635e621fb8d) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:23.729963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8f26d-ec2b-4de7-9deb-90485616968f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:23.731455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8f26d-ec2b-4de7-9deb-90485616968f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:23.738521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 182/380] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:23 2022] GET /identity/v3/role_assignments?group.id=50e7aa331f1e49e2a0870ced5186e649&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:24.245707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48289d6-fd5e-44a5-8707-2750d8cce20d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:24.246217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48289d6-fd5e-44a5-8707-2750d8cce20d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:24.246289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48289d6-fd5e-44a5-8707-2750d8cce20d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:24.247638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 199/381] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:24.253289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135cd27-34dc-4a0b-94e3-b443d464a481 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:24.254156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135cd27-34dc-4a0b-94e3-b443d464a481 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:24.254644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135cd27-34dc-4a0b-94e3-b443d464a481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:24.273748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7135cd27-34dc-4a0b-94e3-b443d464a481 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:24.275892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7135cd27-34dc-4a0b-94e3-b443d464a481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:24.363887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 183/382] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:24.400527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48289d6-fd5e-44a5-8707-2750d8cce20d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:24.535714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:24.546291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48289d6-fd5e-44a5-8707-2750d8cce20d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:24.600493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b1acc-a2a1-440c-ac1d-5119afb0f41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:24.601306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b1acc-a2a1-440c-ac1d-5119afb0f41a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:24.601590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b1acc-a2a1-440c-ac1d-5119afb0f41a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:24.601863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b1acc-a2a1-440c-ac1d-5119afb0f41a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:24.622292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b1acc-a2a1-440c-ac1d-5119afb0f41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-941b1acc-a2a1-440c-ac1d-5119afb0f41a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:24.630805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:24.631663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:24.632479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:24.640458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 200/383] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:24.645278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135cd27-34dc-4a0b-94e3-b443d464a481 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:24.651109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e247f-5168-46e7-8721-24b7a9f68b8c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:24.651717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e247f-5168-46e7-8721-24b7a9f68b8c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:24.652214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e247f-5168-46e7-8721-24b7a9f68b8c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:24.653824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 201/384] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:24.659458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42321776-6cb6-42f5-a554-576bcc7a3373 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:24.659975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42321776-6cb6-42f5-a554-576bcc7a3373 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:24.660515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42321776-6cb6-42f5-a554-576bcc7a3373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:24.688115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-42321776-6cb6-42f5-a554-576bcc7a3373 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:24.689660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-42321776-6cb6-42f5-a554-576bcc7a3373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:24.702505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:24.758198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135cd27-34dc-4a0b-94e3-b443d464a481 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:24.803656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:24.804711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:24.805187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:24.805834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:24.807088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:24.807608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:24.810565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:24.827634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 184/385] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:24.833804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:24.834503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 202/386] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:24.885701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42321776-6cb6-42f5-a554-576bcc7a3373 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:25.010220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f9588-6963-4be4-853f-e9d60fff9f7d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:25.032628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:25.060035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22403d3d-6123-4248-b776-97d04fd2d24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:25.060853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22403d3d-6123-4248-b776-97d04fd2d24e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.061194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22403d3d-6123-4248-b776-97d04fd2d24e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.061469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22403d3d-6123-4248-b776-97d04fd2d24e None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.062716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22403d3d-6123-4248-b776-97d04fd2d24e None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:25.064183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22403d3d-6123-4248-b776-97d04fd2d24e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:25.078094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 185/387] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:24 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:25.079014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22403d3d-6123-4248-b776-97d04fd2d24e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:25.100512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:25.131239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42321776-6cb6-42f5-a554-576bcc7a3373 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:25.171723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba5b86-8f09-4706-acd8-cbff97251c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:25.172603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba5b86-8f09-4706-acd8-cbff97251c26 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.172890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba5b86-8f09-4706-acd8-cbff97251c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.173169 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba5b86-8f09-4706-acd8-cbff97251c26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.185717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba5b86-8f09-4706-acd8-cbff97251c26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d1ba5b86-8f09-4706-acd8-cbff97251c26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:25.195296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:25.197048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:25.198403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 203/388] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:25.211075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba5b86-8f09-4706-acd8-cbff97251c26 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:25.285026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22403d3d-6123-4248-b776-97d04fd2d24e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:25.325778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46eb6e-48fc-4c1b-ac4a-4d72c6c67463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:25.327345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46eb6e-48fc-4c1b-ac4a-4d72c6c67463 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.327730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46eb6e-48fc-4c1b-ac4a-4d72c6c67463 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.328270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46eb6e-48fc-4c1b-ac4a-4d72c6c67463 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.329830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46eb6e-48fc-4c1b-ac4a-4d72c6c67463 None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:25.331540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46eb6e-48fc-4c1b-ac4a-4d72c6c67463 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:25.334177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 186/389] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:25 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:25.335716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc9daa-5388-40fb-a56a-9c059cde8a87 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.336282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc9daa-5388-40fb-a56a-9c059cde8a87 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.337381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc9daa-5388-40fb-a56a-9c059cde8a87 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.339456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 187/390] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:25.349445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc9daa-5388-40fb-a56a-9c059cde8a87 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:25.361910 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba5b86-8f09-4706-acd8-cbff97251c26 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:25.400281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1882f-a58f-4637-a0ca-da2aa08861ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:25.401379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1882f-a58f-4637-a0ca-da2aa08861ec None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.401860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1882f-a58f-4637-a0ca-da2aa08861ec None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.402423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1882f-a58f-4637-a0ca-da2aa08861ec None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.403303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:25.404363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1882f-a58f-4637-a0ca-da2aa08861ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:25.404930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1882f-a58f-4637-a0ca-da2aa08861ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:25.406577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1882f-a58f-4637-a0ca-da2aa08861ec None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:25.418052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 204/391] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:25.419824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e134a-53dc-4353-8b1f-c71b601465d7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.420470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e134a-53dc-4353-8b1f-c71b601465d7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.420973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e134a-53dc-4353-8b1f-c71b601465d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.427755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:25.443847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a33e134a-53dc-4353-8b1f-c71b601465d7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:25.445555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a33e134a-53dc-4353-8b1f-c71b601465d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:25.469074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc9daa-5388-40fb-a56a-9c059cde8a87 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:25.508864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5d60f-e720-4955-b045-5be087d4b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:25.510154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5d60f-e720-4955-b045-5be087d4b117 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.510582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5d60f-e720-4955-b045-5be087d4b117 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.511230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5d60f-e720-4955-b045-5be087d4b117 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d/groups/50e7aa331f1e49e2a0870ced5186e649/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.513547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5d60f-e720-4955-b045-5be087d4b117 None admin] RBAC: Authorizing `identity:create_grant(project_id=5229c8d118a3441fa4e48635e621fb8d, group_id=50e7aa331f1e49e2a0870ced5186e649, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:25.515328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5d60f-e720-4955-b045-5be087d4b117 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:25.532944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 188/392] 10.222.0.32 () {58 vars in 1442 bytes} [Mon May 30 11:50:25 2022] PUT /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d/groups/50e7aa331f1e49e2a0870ced5186e649/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:25.533959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5d60f-e720-4955-b045-5be087d4b117 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:25.537604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 205/393] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:25.596093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e134a-53dc-4353-8b1f-c71b601465d7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:25.675576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5d60f-e720-4955-b045-5be087d4b117 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:25.683843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:25.716796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc63623-c174-4736-b03f-7af61ee97401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:25.717671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc63623-c174-4736-b03f-7af61ee97401 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.717956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc63623-c174-4736-b03f-7af61ee97401 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.718235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc63623-c174-4736-b03f-7af61ee97401 None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.719489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc63623-c174-4736-b03f-7af61ee97401 None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:25.721370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc63623-c174-4736-b03f-7af61ee97401 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:25.723601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 189/394] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:25 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:25.733537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc63623-c174-4736-b03f-7af61ee97401 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:25.805676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e134a-53dc-4353-8b1f-c71b601465d7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:25.847983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0aacc-175c-4716-9c1e-8c08ec6b700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:25.849199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0aacc-175c-4716-9c1e-8c08ec6b700a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.849684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0aacc-175c-4716-9c1e-8c08ec6b700a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.850272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0aacc-175c-4716-9c1e-8c08ec6b700a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.853282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:25.864591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0aacc-175c-4716-9c1e-8c08ec6b700a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-aaa0aacc-175c-4716-9c1e-8c08ec6b700a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:25.873778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:25.874700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:25.875522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:25.878355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 206/395] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:25.887415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0aacc-175c-4716-9c1e-8c08ec6b700a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:25.908591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc63623-c174-4736-b03f-7af61ee97401 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:25.953833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b652-cc15-48c7-ad60-6e1e09f28fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:25.954647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b652-cc15-48c7-ad60-6e1e09f28fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:25.954934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b652-cc15-48c7-ad60-6e1e09f28fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:25.955269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b652-cc15-48c7-ad60-6e1e09f28fe9 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:25.956565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221b652-cc15-48c7-ad60-6e1e09f28fe9 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:25.958019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221b652-cc15-48c7-ad60-6e1e09f28fe9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:25.960207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 190/396] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:25 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:25.974411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b652-cc15-48c7-ad60-6e1e09f28fe9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:26.023586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0aacc-175c-4716-9c1e-8c08ec6b700a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:26.064674 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:26.068054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:26.069839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.070291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.070719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.072411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:26.072883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:26.074606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:26.085434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 207/397] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:26.096880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:26.202460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b652-cc15-48c7-ad60-6e1e09f28fe9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:26.220230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:26.266968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb224e4-ada9-4c6c-8df5-5714b7b8701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:26.268412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb224e4-ada9-4c6c-8df5-5714b7b8701a None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.268838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb224e4-ada9-4c6c-8df5-5714b7b8701a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.269256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb224e4-ada9-4c6c-8df5-5714b7b8701a None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/groups/50e7aa331f1e49e2a0870ced5186e649/roles/a511e28f566a49b4bacfbd0325aa44ed` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.271702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb224e4-ada9-4c6c-8df5-5714b7b8701a None admin] RBAC: Authorizing `identity:create_grant(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7, group_id=50e7aa331f1e49e2a0870ced5186e649, role_id=a511e28f566a49b4bacfbd0325aa44ed)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:26.273427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb224e4-ada9-4c6c-8df5-5714b7b8701a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:26.295155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 191/398] 10.222.0.32 () {58 vars in 1442 bytes} [Mon May 30 11:50:25 2022] PUT /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/groups/50e7aa331f1e49e2a0870ced5186e649/roles/a511e28f566a49b4bacfbd0325aa44ed => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 11:50:26.309624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec82e-71c5-479e-8663-00d18eea826d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:26.351041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97107828-ae5d-4d50-8775-c5a8d887bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:26.352926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97107828-ae5d-4d50-8775-c5a8d887bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.353420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97107828-ae5d-4d50-8775-c5a8d887bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.353842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97107828-ae5d-4d50-8775-c5a8d887bdd3 None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.355418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97107828-ae5d-4d50-8775-c5a8d887bdd3 None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:26.357145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97107828-ae5d-4d50-8775-c5a8d887bdd3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:26.359368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 208/399] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:26 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:26.365244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb224e4-ada9-4c6c-8df5-5714b7b8701a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:26.490594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb224e4-ada9-4c6c-8df5-5714b7b8701a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:26.493359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:26.535422 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fa1eb-efef-4235-bea1-7d718b607592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:26.536589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa1eb-efef-4235-bea1-7d718b607592 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.537056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa1eb-efef-4235-bea1-7d718b607592 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.537476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa1eb-efef-4235-bea1-7d718b607592 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.541360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fa1eb-efef-4235-bea1-7d718b607592 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:26.543031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fa1eb-efef-4235-bea1-7d718b607592 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:26.545026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 192/400] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:26 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:26.550959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97107828-ae5d-4d50-8775-c5a8d887bdd3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:26.606438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91404-69b2-48ff-a397-bb10d840ffaf None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.606740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91404-69b2-48ff-a397-bb10d840ffaf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.607006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e91404-69b2-48ff-a397-bb10d840ffaf None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.608940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 193/401] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:50:26.622476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022124d0-675c-49d3-9a92-580179223b24 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.622801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022124d0-675c-49d3-9a92-580179223b24 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.623583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022124d0-675c-49d3-9a92-580179223b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.651841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-022124d0-675c-49d3-9a92-580179223b24 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:26.653534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-022124d0-675c-49d3-9a92-580179223b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:26.675824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97107828-ae5d-4d50-8775-c5a8d887bdd3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:26.706384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:26.728438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fde3c7-9097-4dd6-9c01-e91ccd63a37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:26.729338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fde3c7-9097-4dd6-9c01-e91ccd63a37b None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.729627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fde3c7-9097-4dd6-9c01-e91ccd63a37b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.729907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fde3c7-9097-4dd6-9c01-e91ccd63a37b None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/groups/50e7aa331f1e49e2a0870ced5186e649/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.734168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fde3c7-9097-4dd6-9c01-e91ccd63a37b None admin] RBAC: Authorizing `identity:create_grant(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7, group_id=50e7aa331f1e49e2a0870ced5186e649, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:26.736720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fde3c7-9097-4dd6-9c01-e91ccd63a37b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:26.759346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 209/402] 10.222.0.32 () {58 vars in 1442 bytes} [Mon May 30 11:50:26 2022] PUT /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7/groups/50e7aa331f1e49e2a0870ced5186e649/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:26.767379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 194/403] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:26.791432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fde3c7-9097-4dd6-9c01-e91ccd63a37b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:26.832216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc02b1-6b5d-45c6-9e81-8d269695a165 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.832527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc02b1-6b5d-45c6-9e81-8d269695a165 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.832868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc02b1-6b5d-45c6-9e81-8d269695a165 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.834541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 195/404] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:26.845927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2ebf-c91f-4b25-9bf0-08191053b986 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:26.846253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2ebf-c91f-4b25-9bf0-08191053b986 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:26.846489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c2ebf-c91f-4b25-9bf0-08191053b986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:26.872021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-448c2ebf-c91f-4b25-9bf0-08191053b986 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:26.875099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-448c2ebf-c91f-4b25-9bf0-08191053b986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:26.995311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 196/405] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:27.000734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fde3c7-9097-4dd6-9c01-e91ccd63a37b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:27.018594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c2ebf-c91f-4b25-9bf0-08191053b986 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:27.055453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148d2ec-ccd2-42b5-a4ca-03a477a9fa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:27.056424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148d2ec-ccd2-42b5-a4ca-03a477a9fa87 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:27.056758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148d2ec-ccd2-42b5-a4ca-03a477a9fa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:27.057039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148d2ec-ccd2-42b5-a4ca-03a477a9fa87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:27.057728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:27.069952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148d2ec-ccd2-42b5-a4ca-03a477a9fa87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6148d2ec-ccd2-42b5-a4ca-03a477a9fa87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:27.078691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:27.079887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:27.080934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 210/406] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:27.093208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148d2ec-ccd2-42b5-a4ca-03a477a9fa87 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:27.171474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c2ebf-c91f-4b25-9bf0-08191053b986 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:27.214771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:27.216045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c1291-4a04-40bf-852a-c6e73d1e655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:27.217263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c1291-4a04-40bf-852a-c6e73d1e655f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:27.217697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c1291-4a04-40bf-852a-c6e73d1e655f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:27.218187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c1291-4a04-40bf-852a-c6e73d1e655f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:27.231047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c1291-4a04-40bf-852a-c6e73d1e655f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7d8c1291-4a04-40bf-852a-c6e73d1e655f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:27.241989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:27.244022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:27.245086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:27.247580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 197/407] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:27.253805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c1291-4a04-40bf-852a-c6e73d1e655f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:27.356514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:27.370559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c1291-4a04-40bf-852a-c6e73d1e655f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:27.373832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148d2ec-ccd2-42b5-a4ca-03a477a9fa87 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:27.438521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:27.439202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:27.439489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:27.439770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:27.440582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:27.441226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:27.442575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:27.444828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:27.445725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:27.446485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:27.447096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:27.448041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:27.448572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:27.450145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:27.456334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 198/408] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:27.460179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:27.474922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 211/409] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:27.480696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:27.578965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fce117-66c1-48aa-8509-03f043a87eb9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:27.614617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307e0d8-edc8-492a-a3ed-5738b616c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:27.615410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e0d8-edc8-492a-a3ed-5738b616c277 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:27.615697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e0d8-edc8-492a-a3ed-5738b616c277 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:27.615975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e0d8-edc8-492a-a3ed-5738b616c277 None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:27.617220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307e0d8-edc8-492a-a3ed-5738b616c277 None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:27.619534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307e0d8-edc8-492a-a3ed-5738b616c277 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:27.621516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 199/410] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:27 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:27.626931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307e0d8-edc8-492a-a3ed-5738b616c277 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:27.672584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:27.736263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307e0d8-edc8-492a-a3ed-5738b616c277 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:27.774981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f8413-1113-4fbb-8ef3-adb37372531e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:27.775862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f8413-1113-4fbb-8ef3-adb37372531e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:27.776146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f8413-1113-4fbb-8ef3-adb37372531e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:27.776486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f8413-1113-4fbb-8ef3-adb37372531e None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:27.777990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f8413-1113-4fbb-8ef3-adb37372531e None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:27.779449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f8413-1113-4fbb-8ef3-adb37372531e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:27.782020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 200/411] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:27 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:27.787651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f8413-1113-4fbb-8ef3-adb37372531e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:27.826181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490cab6-9023-4f8d-bdc4-98688a300bcb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:27.834364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:27.877545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87438e7-0ae1-421b-8854-a4d1fd0cca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:27.878668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87438e7-0ae1-421b-8854-a4d1fd0cca97 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:27.878961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87438e7-0ae1-421b-8854-a4d1fd0cca97 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:27.879243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87438e7-0ae1-421b-8854-a4d1fd0cca97 None admin] PATH_INFO: `/v3/groups/1c182bca33b2411e8eee0e74f3b7c6e5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:27.880699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87438e7-0ae1-421b-8854-a4d1fd0cca97 None admin] RBAC: Authorizing `identity:get_group(group_id=1c182bca33b2411e8eee0e74f3b7c6e5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:27.882761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87438e7-0ae1-421b-8854-a4d1fd0cca97 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:27.887559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 212/412] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:27 2022] GET /identity/v3/groups/1c182bca33b2411e8eee0e74f3b7c6e5 => generated 229 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:27.893150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87438e7-0ae1-421b-8854-a4d1fd0cca97 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:27.912977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:27.928099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f8413-1113-4fbb-8ef3-adb37372531e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:27.965101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1058984-2771-4980-92b0-3c11033acee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:27.966010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1058984-2771-4980-92b0-3c11033acee4 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:27.966301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1058984-2771-4980-92b0-3c11033acee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:27.966587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1058984-2771-4980-92b0-3c11033acee4 None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d/groups/50e7aa331f1e49e2a0870ced5186e649/roles/a511e28f566a49b4bacfbd0325aa44ed` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:27.969075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1058984-2771-4980-92b0-3c11033acee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5229c8d118a3441fa4e48635e621fb8d, group_id=50e7aa331f1e49e2a0870ced5186e649, role_id=a511e28f566a49b4bacfbd0325aa44ed)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:27.970533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1058984-2771-4980-92b0-3c11033acee4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:27.995723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 201/413] 10.222.0.32 () {58 vars in 1442 bytes} [Mon May 30 11:50:27 2022] PUT /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d/groups/50e7aa331f1e49e2a0870ced5186e649/roles/a511e28f566a49b4bacfbd0325aa44ed => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 11:50:28.079956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87438e7-0ae1-421b-8854-a4d1fd0cca97 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:28.121997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4c4a3-9325-4cdf-91db-a347e4680ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:28.122927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c4a3-9325-4cdf-91db-a347e4680ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:28.123216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c4a3-9325-4cdf-91db-a347e4680ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:28.123501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c4a3-9325-4cdf-91db-a347e4680ef2 None admin] PATH_INFO: `/v3/projects/41eaf29f1f484186a95e357823daad72` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:28.124899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4c4a3-9325-4cdf-91db-a347e4680ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=41eaf29f1f484186a95e357823daad72)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:28.126456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4c4a3-9325-4cdf-91db-a347e4680ef2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:28.128849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 213/414] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:27 2022] GET /identity/v3/projects/41eaf29f1f484186a95e357823daad72 => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:28.136606 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1058984-2771-4980-92b0-3c11033acee4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:28.289123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1058984-2771-4980-92b0-3c11033acee4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:28.326261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45263c62-71f5-4f31-99cd-aa7755e2c203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:28.327161 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45263c62-71f5-4f31-99cd-aa7755e2c203 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:28.327446 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45263c62-71f5-4f31-99cd-aa7755e2c203 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:28.327725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45263c62-71f5-4f31-99cd-aa7755e2c203 None admin] PATH_INFO: `/v3/projects/41eaf29f1f484186a95e357823daad72/groups/1c182bca33b2411e8eee0e74f3b7c6e5/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:28.329854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45263c62-71f5-4f31-99cd-aa7755e2c203 None admin] RBAC: Authorizing `identity:create_grant(project_id=41eaf29f1f484186a95e357823daad72, group_id=1c182bca33b2411e8eee0e74f3b7c6e5, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:28.331311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45263c62-71f5-4f31-99cd-aa7755e2c203 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:28.346966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 202/415] 10.222.0.32 () {58 vars in 1442 bytes} [Mon May 30 11:50:28 2022] PUT /identity/v3/projects/41eaf29f1f484186a95e357823daad72/groups/1c182bca33b2411e8eee0e74f3b7c6e5/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:28.612148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee400-8a58-4d9b-b14e-5fe13aa5aa9d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:28.612674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee400-8a58-4d9b-b14e-5fe13aa5aa9d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:28.613172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee400-8a58-4d9b-b14e-5fe13aa5aa9d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:28.615711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 214/416] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:28.622781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bce15-9271-4331-b65c-a3553bafe8f2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:28.623597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bce15-9271-4331-b65c-a3553bafe8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:28.623808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bce15-9271-4331-b65c-a3553bafe8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:28.648797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b51bce15-9271-4331-b65c-a3553bafe8f2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:28.650778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b51bce15-9271-4331-b65c-a3553bafe8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:28.653321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:28.740508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 203/417] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:28.769654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aee400-8a58-4d9b-b14e-5fe13aa5aa9d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:28.817923 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:28.900825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aee400-8a58-4d9b-b14e-5fe13aa5aa9d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:28.953454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5751ec1-55e5-4337-bfa1-53e82d8f3784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:28.954558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751ec1-55e5-4337-bfa1-53e82d8f3784 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:28.954989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751ec1-55e5-4337-bfa1-53e82d8f3784 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:28.955412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751ec1-55e5-4337-bfa1-53e82d8f3784 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:28.967619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5751ec1-55e5-4337-bfa1-53e82d8f3784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a5751ec1-55e5-4337-bfa1-53e82d8f3784 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:28.975635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:28.976757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:28.978330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:28.980744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 215/418] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:28.986378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bce15-9271-4331-b65c-a3553bafe8f2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:29.141128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bce15-9271-4331-b65c-a3553bafe8f2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:29.195046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:29.195257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2481c0-3fe5-4778-b99a-a855668f679b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.195468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2481c0-3fe5-4778-b99a-a855668f679b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.196480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2481c0-3fe5-4778-b99a-a855668f679b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.196795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.197084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.197364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.198139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:29.198455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:29.199081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 216/419] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:29.203194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:29.204857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe05d4d-1bcd-4265-a5dc-553e40249efd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.205293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe05d4d-1bcd-4265-a5dc-553e40249efd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.205714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe05d4d-1bcd-4265-a5dc-553e40249efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.207202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:29.216445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 204/420] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:29.221996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:29.237927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bfe05d4d-1bcd-4265-a5dc-553e40249efd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:29.239476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bfe05d4d-1bcd-4265-a5dc-553e40249efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:29.323205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 217/421] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:29.370842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe05d4d-1bcd-4265-a5dc-553e40249efd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:29.411303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf748552-5a19-440a-96fc-53d052fa3569 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:29.425945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:29.488664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585bead-24e8-4004-8450-6ba2cf122c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:29.489630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585bead-24e8-4004-8450-6ba2cf122c7d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.490093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585bead-24e8-4004-8450-6ba2cf122c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.490547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585bead-24e8-4004-8450-6ba2cf122c7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.527499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe05d4d-1bcd-4265-a5dc-553e40249efd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:29.578107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a222ba-dd2b-46ed-80f5-4f0975c797f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:29.579467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a222ba-dd2b-46ed-80f5-4f0975c797f7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.579917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a222ba-dd2b-46ed-80f5-4f0975c797f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.580381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a222ba-dd2b-46ed-80f5-4f0975c797f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.601046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a222ba-dd2b-46ed-80f5-4f0975c797f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:29.601538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585bead-24e8-4004-8450-6ba2cf122c7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:29.603811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585bead-24e8-4004-8450-6ba2cf122c7d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:29.612728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e4a222ba-dd2b-46ed-80f5-4f0975c797f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:29.621751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:29.622907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:29.623885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:29.626197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 218/422] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:29.631435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a222ba-dd2b-46ed-80f5-4f0975c797f7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:29.749183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 205/423] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 30 11:50:29.750457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25e0f4-b504-447c-a861-e3907e2e97ab None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.750748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25e0f4-b504-447c-a861-e3907e2e97ab None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.751028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25e0f4-b504-447c-a861-e3907e2e97ab None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.756948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 206/424] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:29.757738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25e0f4-b504-447c-a861-e3907e2e97ab None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:29.759156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:29.772585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a222ba-dd2b-46ed-80f5-4f0975c797f7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:29.810980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e07b0-76ed-4cf7-9cb9-7aa1eaa0fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:29.811996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e07b0-76ed-4cf7-9cb9-7aa1eaa0fda6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.812522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e07b0-76ed-4cf7-9cb9-7aa1eaa0fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.812999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e07b0-76ed-4cf7-9cb9-7aa1eaa0fda6 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.813960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e07b0-76ed-4cf7-9cb9-7aa1eaa0fda6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:29.814487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e07b0-76ed-4cf7-9cb9-7aa1eaa0fda6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:29.816077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e07b0-76ed-4cf7-9cb9-7aa1eaa0fda6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:29.829851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 219/425] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:29.831565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c372d-cd6b-4b15-9dd0-7e6d44a24c4d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.832058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c372d-cd6b-4b15-9dd0-7e6d44a24c4d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.832603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c372d-cd6b-4b15-9dd0-7e6d44a24c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.850121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b89c372d-cd6b-4b15-9dd0-7e6d44a24c4d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:29.851600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b89c372d-cd6b-4b15-9dd0-7e6d44a24c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:29.887399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:29.893015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25e0f4-b504-447c-a861-e3907e2e97ab None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:29.927867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 220/426] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:29.929403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c372d-cd6b-4b15-9dd0-7e6d44a24c4d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:29.938878 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2efa8c-2ddf-4526-aeb0-d51f53088cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:29.939766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2efa8c-2ddf-4526-aeb0-d51f53088cdc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:29.940053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2efa8c-2ddf-4526-aeb0-d51f53088cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:29.940369 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2efa8c-2ddf-4526-aeb0-d51f53088cdc None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:29.943084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2efa8c-2ddf-4526-aeb0-d51f53088cdc None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:29.944554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2efa8c-2ddf-4526-aeb0-d51f53088cdc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:29.952107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 207/427] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:29 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:29.952828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2efa8c-2ddf-4526-aeb0-d51f53088cdc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:30.029162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:30.047185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89c372d-cd6b-4b15-9dd0-7e6d44a24c4d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:30.079028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2efa8c-2ddf-4526-aeb0-d51f53088cdc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:30.083888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13519ac-7589-47b7-b152-54451ff9ba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:30.085024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13519ac-7589-47b7-b152-54451ff9ba04 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.085544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13519ac-7589-47b7-b152-54451ff9ba04 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.086033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13519ac-7589-47b7-b152-54451ff9ba04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.117578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7d863-a77a-4a15-b231-84fdf19c568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:30.118322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d863-a77a-4a15-b231-84fdf19c568a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.118613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d863-a77a-4a15-b231-84fdf19c568a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.118894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d863-a77a-4a15-b231-84fdf19c568a None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.120171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7d863-a77a-4a15-b231-84fdf19c568a None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:30.121765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7d863-a77a-4a15-b231-84fdf19c568a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:30.128195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 208/428] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:29 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:30.128195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7d863-a77a-4a15-b231-84fdf19c568a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:30.175567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:30.178076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13519ac-7589-47b7-b152-54451ff9ba04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:30.179934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13519ac-7589-47b7-b152-54451ff9ba04 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:30.255413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7d863-a77a-4a15-b231-84fdf19c568a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:30.276792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:30.294911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa54de-4b5f-4faf-8dfa-0befa26d1a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:30.295733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa54de-4b5f-4faf-8dfa-0befa26d1a38 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.296058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 221/429] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:30.296178 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa54de-4b5f-4faf-8dfa-0befa26d1a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.296460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa54de-4b5f-4faf-8dfa-0befa26d1a38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.297628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13519ac-7589-47b7-b152-54451ff9ba04 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:30.308495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa54de-4b5f-4faf-8dfa-0befa26d1a38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-befa54de-4b5f-4faf-8dfa-0befa26d1a38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:30.316518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:30.317413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:30.318224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 209/430] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:30.318881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa54de-4b5f-4faf-8dfa-0befa26d1a38 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:30.413785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13519ac-7589-47b7-b152-54451ff9ba04 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:30.427595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:30.434358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa54de-4b5f-4faf-8dfa-0befa26d1a38 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:30.454872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:30.455942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.456497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.457127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.458244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] RBAC: Adding query filter params (group.id=50e7aa331f1e49e2a0870ced5186e649, role.id=932e429fa28248f285663a90fc880876, scope.project.id=5229c8d118a3441fa4e48635e621fb8d) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:30.458949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:30.460984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:30.469493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 222/431] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:30 2022] GET /identity/v3/role_assignments?group.id=50e7aa331f1e49e2a0870ced5186e649&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=932e429fa28248f285663a90fc880876 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:30.470634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:30.481495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e36c68-a677-46dd-84f1-375adc5fd6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:30.482446 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36c68-a677-46dd-84f1-375adc5fd6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.482733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36c68-a677-46dd-84f1-375adc5fd6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.483014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36c68-a677-46dd-84f1-375adc5fd6b4 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.485798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e36c68-a677-46dd-84f1-375adc5fd6b4 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:30.487329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e36c68-a677-46dd-84f1-375adc5fd6b4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:30.489338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 210/432] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:30 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:30.495567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e36c68-a677-46dd-84f1-375adc5fd6b4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:30.532375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:30.591239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b9bc1-1e48-4da6-9878-416b94f7ab8a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:30.630622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:30.631511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.631906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.632332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.633746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:30.634185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:30.635870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:30.646753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 223/433] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:50:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:30.651907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:30.749067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e36c68-a677-46dd-84f1-375adc5fd6b4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:30.793123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eeaa9a-bced-4817-a6c0-69cf0049a0b3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:30.798687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528c5da-08d2-43e9-85f6-6da43586d1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:30.799572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528c5da-08d2-43e9-85f6-6da43586d1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.799858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528c5da-08d2-43e9-85f6-6da43586d1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.800135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528c5da-08d2-43e9-85f6-6da43586d1c7 None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.801384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528c5da-08d2-43e9-85f6-6da43586d1c7 None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:30.802840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528c5da-08d2-43e9-85f6-6da43586d1c7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:30.805515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 211/434] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:30 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:30.806822 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c102e9-6357-49de-bc01-348562366b20 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.807111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c102e9-6357-49de-bc01-348562366b20 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.807395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c102e9-6357-49de-bc01-348562366b20 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.808707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 212/435] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:30.809212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:30.814907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21898f8-ed4f-443c-a749-bc9e320c4c19 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.815193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21898f8-ed4f-443c-a749-bc9e320c4c19 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.815474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21898f8-ed4f-443c-a749-bc9e320c4c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.835845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f21898f8-ed4f-443c-a749-bc9e320c4c19 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:30.837932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f21898f8-ed4f-443c-a749-bc9e320c4c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:30.842730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab35664-1439-46cd-b96a-8ba19651afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:30.843345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab35664-1439-46cd-b96a-8ba19651afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:30.843633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab35664-1439-46cd-b96a-8ba19651afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:30.843911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab35664-1439-46cd-b96a-8ba19651afb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:30.867323 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:30.917958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 213/436] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:30.918960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21898f8-ed4f-443c-a749-bc9e320c4c19 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:30.955626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab35664-1439-46cd-b96a-8ba19651afb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:30.958076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab35664-1439-46cd-b96a-8ba19651afb0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:31.029188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21898f8-ed4f-443c-a749-bc9e320c4c19 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.064065 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.064756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.065046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.065340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.066165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] RBAC: Adding query filter params (group.id=50e7aa331f1e49e2a0870ced5186e649, role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:31.066483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:31.067926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:31.068630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 224/437] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:31.069774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab35664-1439-46cd-b96a-8ba19651afb0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.078013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 214/438] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:30 2022] GET /identity/v3/role_assignments?group.id=50e7aa331f1e49e2a0870ced5186e649&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:31.079070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.090235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:31.127302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:31.208426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab35664-1439-46cd-b96a-8ba19651afb0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.230208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529ae3d-1e59-4824-9721-caf5098e79f0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.254758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7fed4-ffc0-4bee-92f0-2a98a3ff402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.255588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7fed4-ffc0-4bee-92f0-2a98a3ff402f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.255877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7fed4-ffc0-4bee-92f0-2a98a3ff402f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.256189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7fed4-ffc0-4bee-92f0-2a98a3ff402f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.269264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7fed4-ffc0-4bee-92f0-2a98a3ff402f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:31.272409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86345ee-bd05-4fc7-8b24-e506f37acee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.273299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86345ee-bd05-4fc7-8b24-e506f37acee5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.273586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86345ee-bd05-4fc7-8b24-e506f37acee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.274394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86345ee-bd05-4fc7-8b24-e506f37acee5 None admin] PATH_INFO: `/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.275649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86345ee-bd05-4fc7-8b24-e506f37acee5 None admin] RBAC: Authorizing `identity:get_project(project_id=cf52a898a25b4ba1a56dcdbb3174e7a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:31.277134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86345ee-bd05-4fc7-8b24-e506f37acee5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-61c7fed4-ffc0-4bee-92f0-2a98a3ff402f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:31.278746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:31.279606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:31.285249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:31.286003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 215/439] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/projects/cf52a898a25b4ba1a56dcdbb3174e7a7 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:31.286086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 225/440] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:31.286163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f225867-ef63-4c37-9050-bf14d0e254ab None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.286163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f225867-ef63-4c37-9050-bf14d0e254ab None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.286163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f225867-ef63-4c37-9050-bf14d0e254ab None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.286163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 216/441] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:31.291228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7fed4-ffc0-4bee-92f0-2a98a3ff402f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.292534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f225867-ef63-4c37-9050-bf14d0e254ab None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.370106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:31.400782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:31.403724 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7fed4-ffc0-4bee-92f0-2a98a3ff402f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.408875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f225867-ef63-4c37-9050-bf14d0e254ab None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.438027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.438692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.438973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.439243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.439980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:31.440317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:31.441700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:31.441826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab141c-2d98-4d72-ae12-ac7a5fa866a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.442578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab141c-2d98-4d72-ae12-ac7a5fa866a9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.442861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab141c-2d98-4d72-ae12-ac7a5fa866a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.443134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab141c-2d98-4d72-ae12-ac7a5fa866a9 None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.444318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab141c-2d98-4d72-ae12-ac7a5fa866a9 None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:31.446269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab141c-2d98-4d72-ae12-ac7a5fa866a9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:31.447929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 217/442] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:31.452732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d014-6a45-49c1-af28-814c8d11095f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.452732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d014-6a45-49c1-af28-814c8d11095f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.452732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d014-6a45-49c1-af28-814c8d11095f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.452958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 226/443] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:31.453741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.466886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d2a6d014-6a45-49c1-af28-814c8d11095f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:31.468081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d2a6d014-6a45-49c1-af28-814c8d11095f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:31.533990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 218/444] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:31.534835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6d014-6a45-49c1-af28-814c8d11095f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.556184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df98ef-3f1a-4923-ae6f-5d1292718cb6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.590958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:31.596875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.597543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.597829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.598109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.598953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] RBAC: Adding query filter params (group.id=50e7aa331f1e49e2a0870ced5186e649, role.id=932e429fa28248f285663a90fc880876, scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:31.599267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:31.600755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:31.609289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 227/445] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/role_assignments?group.id=50e7aa331f1e49e2a0870ced5186e649&scope.project.id=cf52a898a25b4ba1a56dcdbb3174e7a7&role.id=932e429fa28248f285663a90fc880876 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:31.610710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.631675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:31.646226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6d014-6a45-49c1-af28-814c8d11095f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.685530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3fb21-8e56-4f9c-8363-305377df52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.686135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3fb21-8e56-4f9c-8363-305377df52e6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.686536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3fb21-8e56-4f9c-8363-305377df52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.686835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3fb21-8e56-4f9c-8363-305377df52e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.743859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095ddb1-4505-4fec-a5a5-23af48f2c154 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.769659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3fb21-8e56-4f9c-8363-305377df52e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:31.771179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3fb21-8e56-4f9c-8363-305377df52e6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:31.785067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e14c4-ae5c-4931-a40d-854a3c2d5413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.786146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e14c4-ae5c-4931-a40d-854a3c2d5413 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.786614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e14c4-ae5c-4931-a40d-854a3c2d5413 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.787040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e14c4-ae5c-4931-a40d-854a3c2d5413 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.800513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e14c4-ae5c-4931-a40d-854a3c2d5413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4b1e14c4-ae5c-4931-a40d-854a3c2d5413 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:31.809197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:31.810125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:31.810926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 228/446] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:31.816255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e14c4-ae5c-4931-a40d-854a3c2d5413 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.880501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:31.881016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:31.896567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 219/447] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:31.902768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3fb21-8e56-4f9c-8363-305377df52e6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:31.942982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e14c4-ae5c-4931-a40d-854a3c2d5413 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:31.984915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:31.985593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:31.985878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:31.986159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:31.986914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:31.987222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:31.988659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:31.999771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 229/448] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:32.005575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:32.026905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3fb21-8e56-4f9c-8363-305377df52e6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:32.072788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a17fef-b9b1-4e01-8e33-f8e38dbb856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:32.073665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17fef-b9b1-4e01-8e33-f8e38dbb856a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:32.073946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17fef-b9b1-4e01-8e33-f8e38dbb856a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:32.074225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17fef-b9b1-4e01-8e33-f8e38dbb856a None admin] PATH_INFO: `/v3/projects/5229c8d118a3441fa4e48635e621fb8d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:32.075443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a17fef-b9b1-4e01-8e33-f8e38dbb856a None admin] RBAC: Authorizing `identity:get_project(project_id=5229c8d118a3441fa4e48635e621fb8d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:32.076939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a17fef-b9b1-4e01-8e33-f8e38dbb856a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:32.078636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 220/449] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:31 2022] GET /identity/v3/projects/5229c8d118a3441fa4e48635e621fb8d => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:32.084709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a17fef-b9b1-4e01-8e33-f8e38dbb856a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:32.120960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f325c-a3ba-41d6-bd31-3e14147fec0b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:32.143010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:32.162846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572a8e6-6290-4f04-84d4-d8797d36e0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:32.163819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572a8e6-6290-4f04-84d4-d8797d36e0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:32.164305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572a8e6-6290-4f04-84d4-d8797d36e0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:32.165549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572a8e6-6290-4f04-84d4-d8797d36e0a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:32.171829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:32.225713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a17fef-b9b1-4e01-8e33-f8e38dbb856a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:32.247612 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572a8e6-6290-4f04-84d4-d8797d36e0a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:32.249171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572a8e6-6290-4f04-84d4-d8797d36e0a7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:32.265873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a89403-c131-430d-9db0-3e977e092ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:32.267441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a89403-c131-430d-9db0-3e977e092ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:32.267843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a89403-c131-430d-9db0-3e977e092ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:32.268360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a89403-c131-430d-9db0-3e977e092ad2 None admin] PATH_INFO: `/v3/groups/50e7aa331f1e49e2a0870ced5186e649` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:32.270645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a89403-c131-430d-9db0-3e977e092ad2 None admin] RBAC: Authorizing `identity:get_group(group_id=50e7aa331f1e49e2a0870ced5186e649)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:32.272109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a89403-c131-430d-9db0-3e977e092ad2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:32.274605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 221/450] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:32 2022] GET /identity/v3/groups/50e7aa331f1e49e2a0870ced5186e649 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:32.281349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a89403-c131-430d-9db0-3e977e092ad2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:32.376789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 230/451] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:32.382291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572a8e6-6290-4f04-84d4-d8797d36e0a7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:32.394318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a89403-c131-430d-9db0-3e977e092ad2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:32.401111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:32.421271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:32.429307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:32.430879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:32.431257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:32.431545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:32.432416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] RBAC: Adding query filter params (group.id=50e7aa331f1e49e2a0870ced5186e649, role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=5229c8d118a3441fa4e48635e621fb8d) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:32.432744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:32.434192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:32.440020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 222/452] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:32 2022] GET /identity/v3/role_assignments?group.id=50e7aa331f1e49e2a0870ced5186e649&scope.project.id=5229c8d118a3441fa4e48635e621fb8d&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:32.488546 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572a8e6-6290-4f04-84d4-d8797d36e0a7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:32.527397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe025a-3734-4373-a74c-dfd92f92173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:32.528314 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe025a-3734-4373-a74c-dfd92f92173c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:32.528594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe025a-3734-4373-a74c-dfd92f92173c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:32.528868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe025a-3734-4373-a74c-dfd92f92173c None admin] PATH_INFO: `/v3/projects/41eaf29f1f484186a95e357823daad72` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:32.530115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe025a-3734-4373-a74c-dfd92f92173c None admin] RBAC: Authorizing `identity:get_project(project_id=41eaf29f1f484186a95e357823daad72)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:32.531582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe025a-3734-4373-a74c-dfd92f92173c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:32.533508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 231/453] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:50:32 2022] GET /identity/v3/projects/41eaf29f1f484186a95e357823daad72 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:32.539414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:32.645920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe829e-c124-46ed-8a02-37cf4594ef9e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:32.682548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d325c-f9bb-45d5-a2c0-55d9539dde82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:32.683415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d325c-f9bb-45d5-a2c0-55d9539dde82 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:32.683700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d325c-f9bb-45d5-a2c0-55d9539dde82 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:32.683977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d325c-f9bb-45d5-a2c0-55d9539dde82 None admin] PATH_INFO: `/v3/groups/1c182bca33b2411e8eee0e74f3b7c6e5` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:32.685181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d325c-f9bb-45d5-a2c0-55d9539dde82 None admin] RBAC: Authorizing `identity:get_group(group_id=1c182bca33b2411e8eee0e74f3b7c6e5)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:32.686625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d325c-f9bb-45d5-a2c0-55d9539dde82 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:32.688272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 223/454] 10.222.0.32 () {56 vars in 1180 bytes} [Mon May 30 11:50:32 2022] GET /identity/v3/groups/1c182bca33b2411e8eee0e74f3b7c6e5 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:32.694482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe025a-3734-4373-a74c-dfd92f92173c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:32.788372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:32.807993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe025a-3734-4373-a74c-dfd92f92173c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:32.965504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc25ded-415f-400f-90dc-759e639d2b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:32.966146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc25ded-415f-400f-90dc-759e639d2b5a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:32.966423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc25ded-415f-400f-90dc-759e639d2b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:32.966693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc25ded-415f-400f-90dc-759e639d2b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:32.967512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc25ded-415f-400f-90dc-759e639d2b5a None admin] RBAC: Adding query filter params (group.id=1c182bca33b2411e8eee0e74f3b7c6e5, role.id=4d21498572b14478a996ac538be5382e, scope.project.id=41eaf29f1f484186a95e357823daad72) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:32.967819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc25ded-415f-400f-90dc-759e639d2b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:32.970080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc25ded-415f-400f-90dc-759e639d2b5a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:32.975074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 232/455] 10.222.0.32 () {56 vars in 1376 bytes} [Mon May 30 11:50:32 2022] GET /identity/v3/role_assignments?group.id=1c182bca33b2411e8eee0e74f3b7c6e5&scope.project.id=41eaf29f1f484186a95e357823daad72&role.id=4d21498572b14478a996ac538be5382e => generated 611 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:35.859172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c840-f4d4-4b33-953a-19cc27cdf719 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:35.860132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c840-f4d4-4b33-953a-19cc27cdf719 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:35.860651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8c840-f4d4-4b33-953a-19cc27cdf719 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:35.862245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 224/456] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:35.873870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cffc1-da17-4320-99f4-b2c37ef071fc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:35.874792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cffc1-da17-4320-99f4-b2c37ef071fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:35.875300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cffc1-da17-4320-99f4-b2c37ef071fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:35.901177 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-568cffc1-da17-4320-99f4-b2c37ef071fc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:35.902745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-568cffc1-da17-4320-99f4-b2c37ef071fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:36.001046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 233/457] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:36.021691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291292-9256-4589-97a8-adf3197e490f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.022370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291292-9256-4589-97a8-adf3197e490f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.022862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291292-9256-4589-97a8-adf3197e490f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.024631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 225/458] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:36.032411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f87a-d21a-46da-b0a1-f12b117f491a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.033040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f87a-d21a-46da-b0a1-f12b117f491a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.033551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f87a-d21a-46da-b0a1-f12b117f491a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.037657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d291292-9256-4589-97a8-adf3197e490f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:36.054192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fa55f87a-d21a-46da-b0a1-f12b117f491a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:36.056088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fa55f87a-d21a-46da-b0a1-f12b117f491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:36.142993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 234/459] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:36.144829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be011-790d-4cd9-ab49-ec704f19404c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.145328 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be011-790d-4cd9-ab49-ec704f19404c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.145800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be011-790d-4cd9-ab49-ec704f19404c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.147613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 235/460] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:36.153714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be789-54cb-494a-9379-49e8d46e46c0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.154336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be789-54cb-494a-9379-49e8d46e46c0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.154860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4be789-54cb-494a-9379-49e8d46e46c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.157736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:36.183153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5d4be789-54cb-494a-9379-49e8d46e46c0 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:36.184377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5d4be789-54cb-494a-9379-49e8d46e46c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:36.187460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:36.207460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d291292-9256-4589-97a8-adf3197e490f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:36.247452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:36.248562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.248991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.249406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.255557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 236/461] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:36.256838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4be789-54cb-494a-9379-49e8d46e46c0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:36.262779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:36.264488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:36.283987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:36.284974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:36.285810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:36.285810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:36.285810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:36.285810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:36.288230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 226/462] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:36.294348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:36.387465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4be789-54cb-494a-9379-49e8d46e46c0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:36.422968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e842fb-2a46-40df-be83-48c04ce77f71 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:36.425213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:36.426021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.426313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.426572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.435876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:36.438263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:36.439068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:36.440786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:36.463922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:36.465016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.465521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.465998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.466988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:36.467528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:36.469204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:36.469378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:36.470925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:36.472560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:36.472560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:36.472560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:36.472560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:36.480575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 237/463] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:36.482581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:36.488455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 227/464] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:36.495112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:36.604509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41366d-0fd4-4f53-ae37-4396dabbcd3a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:36.605946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b34d7-a62f-4e09-8a08-20c53b65c247 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:36.646069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:36.646966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.647160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.647438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.649099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:36.650444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.651444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.651933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.652947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:36.653460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:36.655119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:36.660371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:36.661839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:36.672533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 228/465] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:36.678014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:36.689669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:36.690597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:36.691424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:36.691424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:36.691424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:36.691424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:36.694230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 238/466] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:36.696190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:36.720744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:36.769518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:36.821149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ab3c5-a0e1-47be-9029-06c97a89e76c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:36.841982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391ea39-2783-4f8f-b7d5-5163fd9fabe2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:36.864199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e9ca-068d-4125-85fe-3ee2ed5eb11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:36.864911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e9ca-068d-4125-85fe-3ee2ed5eb11a None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.865157 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e9ca-068d-4125-85fe-3ee2ed5eb11a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.865858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e9ca-068d-4125-85fe-3ee2ed5eb11a None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.866142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708e9ca-068d-4125-85fe-3ee2ed5eb11a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:36.867881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708e9ca-068d-4125-85fe-3ee2ed5eb11a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:36.886351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81411f-21ef-4ab2-a682-a3203247b22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:36.887176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81411f-21ef-4ab2-a682-a3203247b22b None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.887689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81411f-21ef-4ab2-a682-a3203247b22b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.888044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81411f-21ef-4ab2-a682-a3203247b22b None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.888886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81411f-21ef-4ab2-a682-a3203247b22b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:36.890460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81411f-21ef-4ab2-a682-a3203247b22b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:36.894846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 229/467] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:50:36 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:36.895737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e9ca-068d-4125-85fe-3ee2ed5eb11a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:36.928281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 239/468] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:50:36 2022] POST /identity/v3/users => generated 246 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:36.928281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd0ab-6d28-4ab9-8b87-f8c839b33697 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.928281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd0ab-6d28-4ab9-8b87-f8c839b33697 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.928281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd0ab-6d28-4ab9-8b87-f8c839b33697 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.928281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 240/469] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:36.937722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53a7d9-3c62-4f1a-ac38-5b7baf4111f4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:36.939034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53a7d9-3c62-4f1a-ac38-5b7baf4111f4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:36.939331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53a7d9-3c62-4f1a-ac38-5b7baf4111f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:36.974641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0a53a7d9-3c62-4f1a-ac38-5b7baf4111f4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:36.975926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0a53a7d9-3c62-4f1a-ac38-5b7baf4111f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:37.094776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 241/470] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:37.118154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:37.139269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e9ca-068d-4125-85fe-3ee2ed5eb11a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:37.151642 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53a7d9-3c62-4f1a-ac38-5b7baf4111f4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:37.184425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:37.185535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:37.186113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:37.186628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:37.187564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:37.188069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:37.188567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:37.190572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:37.210301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 230/471] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:37.214206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:37.286879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53a7d9-3c62-4f1a-ac38-5b7baf4111f4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:37.337726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05d08d-67f9-4d73-aa61-b7ea6812b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:37.339679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05d08d-67f9-4d73-aa61-b7ea6812b21f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:37.340253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05d08d-67f9-4d73-aa61-b7ea6812b21f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:37.341023 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05d08d-67f9-4d73-aa61-b7ea6812b21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:37.356040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05d08d-67f9-4d73-aa61-b7ea6812b21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:37.357416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b3726-61cb-4036-b9f4-7b2b86a7a525 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:37.358141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05d08d-67f9-4d73-aa61-b7ea6812b21f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ff05d08d-67f9-4d73-aa61-b7ea6812b21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:37.382630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:37.384433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:37.385368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:37.385368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:37.385368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:37.385368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:37.386134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 242/472] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:37.387298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf6314-53fe-4318-9781-0dee011a9c92 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:37.387585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf6314-53fe-4318-9781-0dee011a9c92 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:37.387862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf6314-53fe-4318-9781-0dee011a9c92 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:37.389420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 243/473] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:37.396724 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c512e-f9c8-4a06-bc41-e7c08ed8a593 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:37.397274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c512e-f9c8-4a06-bc41-e7c08ed8a593 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:37.397781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c512e-f9c8-4a06-bc41-e7c08ed8a593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:37.402250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00819684-c4ac-4390-98e2-9252bb57d0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:37.403136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819684-c4ac-4390-98e2-9252bb57d0c8 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:37.403561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819684-c4ac-4390-98e2-9252bb57d0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:37.403981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819684-c4ac-4390-98e2-9252bb57d0c8 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:37.405137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00819684-c4ac-4390-98e2-9252bb57d0c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:37.407652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00819684-c4ac-4390-98e2-9252bb57d0c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:37.418235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-be3c512e-f9c8-4a06-bc41-e7c08ed8a593 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:37.419988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-be3c512e-f9c8-4a06-bc41-e7c08ed8a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:37.436927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 231/474] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:50:37 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:37.437879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00819684-c4ac-4390-98e2-9252bb57d0c8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:37.441987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:37.519734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 244/475] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:37.561877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c512e-f9c8-4a06-bc41-e7c08ed8a593 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:37.577372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00819684-c4ac-4390-98e2-9252bb57d0c8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:37.599528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:37.625149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:37.625842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:37.626129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:37.626412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:37.627174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:37.627485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:37.628958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:37.645131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 232/476] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:37.653259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:37.776832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:37.777914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c512e-f9c8-4a06-bc41-e7c08ed8a593 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:37.825072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259c57b-3f4d-4ba9-be17-2b0eb0033e2c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:37.828211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:37.828906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:37.829202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:37.829484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:37.830243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:37.831653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:37.842234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 30 11:50:37.842234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 30 11:50:37.842234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [parameters: {'id': '4f9f1b72ffb940c089983513a65e995e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 30 11:50:37.842234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 30 11:50:37.862370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return ref.to_dict() May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 30 11:50:37.863333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 30 11:50:37.864288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 30 11:50:37.865222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application c = connection._execute_20( May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 30 11:50:37.866101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 30 11:50:37.866913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'id': '4f9f1b72ffb940c089983513a65e995e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:37.867732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 30 11:50:37.871038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:37.872288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 245/477] 10.222.0.32 () {60 vars in 1131 bytes} [Mon May 30 11:50:37 2022] POST /identity/v3/roles => generated 144 bytes in 303 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:37.873155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:37.885382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0455b-ac62-422f-9335-56b84e46e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:37.886123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0455b-ac62-422f-9335-56b84e46e1dd None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:37.886367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0455b-ac62-422f-9335-56b84e46e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:37.886650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0455b-ac62-422f-9335-56b84e46e1dd None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:37.887370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0455b-ac62-422f-9335-56b84e46e1dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:37.888971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0455b-ac62-422f-9335-56b84e46e1dd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:37.919923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 233/478] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:50:37 2022] POST /identity/v3/users => generated 249 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:37.994412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:38.025361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b9575-a19d-4169-8365-22d0f066f8d0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:38.079530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb444ea5-a99c-44dc-b484-86c1cc127239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:38.080795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb444ea5-a99c-44dc-b484-86c1cc127239 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:38.081294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb444ea5-a99c-44dc-b484-86c1cc127239 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:38.081772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb444ea5-a99c-44dc-b484-86c1cc127239 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:38.098335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb444ea5-a99c-44dc-b484-86c1cc127239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cb444ea5-a99c-44dc-b484-86c1cc127239 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:38.107510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:38.112814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 30 11:50:38.113640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:38.116439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 246/479] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:50:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:38.123012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0455b-ac62-422f-9335-56b84e46e1dd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:38.289618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0455b-ac62-422f-9335-56b84e46e1dd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:38.312637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:38.331344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8788337-7582-4ad3-afbd-1acad618f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:38.331944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788337-7582-4ad3-afbd-1acad618f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:38.332323 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788337-7582-4ad3-afbd-1acad618f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:38.332613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8788337-7582-4ad3-afbd-1acad618f4b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:38.333375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8788337-7582-4ad3-afbd-1acad618f4b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:38.333689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8788337-7582-4ad3-afbd-1acad618f4b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:38.335125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8788337-7582-4ad3-afbd-1acad618f4b6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:38.344891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 234/480] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:39.539823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bf72d-904b-4bca-91d0-94e49ddccac5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:39.541043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bf72d-904b-4bca-91d0-94e49ddccac5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:39.541578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bf72d-904b-4bca-91d0-94e49ddccac5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:39.543515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 247/481] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:39.549121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00310d8a-5e21-40a7-b974-ea4f1f43a0b8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:39.549739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00310d8a-5e21-40a7-b974-ea4f1f43a0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:39.550474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00310d8a-5e21-40a7-b974-ea4f1f43a0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:39.577262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-00310d8a-5e21-40a7-b974-ea4f1f43a0b8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:39.578494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-00310d8a-5e21-40a7-b974-ea4f1f43a0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:39.666150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 235/482] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:39.697417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bf72d-904b-4bca-91d0-94e49ddccac5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:39.823029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b33d0-8722-4995-88d7-64b88988afb6 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:39.823324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b33d0-8722-4995-88d7-64b88988afb6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:39.823717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b33d0-8722-4995-88d7-64b88988afb6 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:39.825262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 236/483] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:39.837926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d67f78-6905-4e5e-8fd8-e68c075c358f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:39.838227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d67f78-6905-4e5e-8fd8-e68c075c358f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:39.838515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d67f78-6905-4e5e-8fd8-e68c075c358f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:39.858927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bf72d-904b-4bca-91d0-94e49ddccac5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:39.884903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a3d67f78-6905-4e5e-8fd8-e68c075c358f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:39.886541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a3d67f78-6905-4e5e-8fd8-e68c075c358f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:39.933942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852074fb-e8a1-43dd-bd29-74f1e5559c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:39.934749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852074fb-e8a1-43dd-bd29-74f1e5559c78 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:39.935029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852074fb-e8a1-43dd-bd29-74f1e5559c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:39.935299 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852074fb-e8a1-43dd-bd29-74f1e5559c78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:39.944069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:39.949323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852074fb-e8a1-43dd-bd29-74f1e5559c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-852074fb-e8a1-43dd-bd29-74f1e5559c78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:39.957162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:39.958092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:39.959596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 248/484] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:39.971572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852074fb-e8a1-43dd-bd29-74f1e5559c78 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.042315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 237/485] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:40.099669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d67f78-6905-4e5e-8fd8-e68c075c358f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.175268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852074fb-e8a1-43dd-bd29-74f1e5559c78 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:40.201793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:40.221229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:40.221906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.222225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.222519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.223301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:40.223720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:40.225089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:40.234880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 249/486] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:40.240316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.306686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:40.348058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4940b-db33-44e9-b71b-411b7fe11e6a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:40.385204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:40.385930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.386217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.386494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.398581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:40.400618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:40.419627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:40.420563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:40.421865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:40.421865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:40.421865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:40.421865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:40.421865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 250/487] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:40.426656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.454212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d67f78-6905-4e5e-8fd8-e68c075c358f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:40.524044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6d7f7-9b42-4394-9a45-66c545c5cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:40.525032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d7f7-9b42-4394-9a45-66c545c5cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.525318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d7f7-9b42-4394-9a45-66c545c5cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.525597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d7f7-9b42-4394-9a45-66c545c5cd2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.536346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6d7f7-9b42-4394-9a45-66c545c5cd2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:40.543204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0bd15-abc0-4f20-a401-06ffa938cb39 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a8e6d7f7-9b42-4394-9a45-66c545c5cd2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:40.545097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:40.545991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:40.547218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 238/488] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:40.548058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594f0da-bd53-4830-8bde-d0f4a7841dfe None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.548348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594f0da-bd53-4830-8bde-d0f4a7841dfe None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.548502 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594f0da-bd53-4830-8bde-d0f4a7841dfe None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.553869 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 239/489] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:40.555322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594f0da-bd53-4830-8bde-d0f4a7841dfe None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.555588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:40.585391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63a124-9b77-4607-966c-15704894f8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:40.586687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63a124-9b77-4607-966c-15704894f8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.589611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63a124-9b77-4607-966c-15704894f8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.589611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63a124-9b77-4607-966c-15704894f8c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.589611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63a124-9b77-4607-966c-15704894f8c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:40.589611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63a124-9b77-4607-966c-15704894f8c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:40.590151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63a124-9b77-4607-966c-15704894f8c2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:40.618858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 251/490] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:40.624537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb9bd3-08e9-4f38-aabd-74d8114314b0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.624828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb9bd3-08e9-4f38-aabd-74d8114314b0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.625114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb9bd3-08e9-4f38-aabd-74d8114314b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.642708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d8fb9bd3-08e9-4f38-aabd-74d8114314b0 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:40.643809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d8fb9bd3-08e9-4f38-aabd-74d8114314b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:40.674520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594f0da-bd53-4830-8bde-d0f4a7841dfe None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:40.714668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:40.715694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.716207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.716736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.717794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:40.718348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:40.719000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:40.721416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:40.722006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 252/491] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:40.722941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb9bd3-08e9-4f38-aabd-74d8114314b0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.733291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 240/492] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:40.738390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.859831 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:40.861463 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d70936-a82f-4a1b-830e-0a4bc7e9a072 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:40.864490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb9bd3-08e9-4f38-aabd-74d8114314b0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:40.903456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:40.904653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.905241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.905811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.907976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:40.909078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:40.909394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:40.910268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:40.915115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:40.917042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:40.925709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:40.927463 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:40.936857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:40.937815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:40.939498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 253/493] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:50:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:40.940056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:40.953930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:40.954848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:40.955720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:40.955720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:40.955720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:40.955720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:40.958133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 241/494] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:40.959230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:40.981780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:41.063767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fe436-0385-4b49-ad37-920699a88afd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.073402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dd968-b3e5-445d-830b-67c0df6d85b0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.102398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aefa52-6603-4349-9049-ad4f3b56f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.103593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aefa52-6603-4349-9049-ad4f3b56f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.104072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aefa52-6603-4349-9049-ad4f3b56f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.104401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aefa52-6603-4349-9049-ad4f3b56f6e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.108855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125f00d-8519-4e37-9ca0-b1ec319c6367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.109487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125f00d-8519-4e37-9ca0-b1ec319c6367 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.109773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125f00d-8519-4e37-9ca0-b1ec319c6367 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.110051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125f00d-8519-4e37-9ca0-b1ec319c6367 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.110890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125f00d-8519-4e37-9ca0-b1ec319c6367 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:41.111206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125f00d-8519-4e37-9ca0-b1ec319c6367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:41.112670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125f00d-8519-4e37-9ca0-b1ec319c6367 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:41.115452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aefa52-6603-4349-9049-ad4f3b56f6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d0aefa52-6603-4349-9049-ad4f3b56f6e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:41.123773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:41.124860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:41.125774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:41.131721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 254/495] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:41.131721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aefa52-6603-4349-9049-ad4f3b56f6e8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:41.155871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:41.157604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 242/496] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:50:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:41.158913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e0c1c-e039-4d63-8344-dd45ba6d80cf None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.159560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e0c1c-e039-4d63-8344-dd45ba6d80cf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.159665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e0c1c-e039-4d63-8344-dd45ba6d80cf None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.161210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 243/497] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:41.161922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e0c1c-e039-4d63-8344-dd45ba6d80cf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:41.247605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aefa52-6603-4349-9049-ad4f3b56f6e8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.258310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:41.289095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.289756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.290045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.290323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.290875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e0c1c-e039-4d63-8344-dd45ba6d80cf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.291157 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:41.291474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:41.292897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:41.310455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 255/498] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:41.310455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:41.332810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd922b-b5a3-4461-90ec-5d02c1d8c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.333485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd922b-b5a3-4461-90ec-5d02c1d8c2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.333769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd922b-b5a3-4461-90ec-5d02c1d8c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.334046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd922b-b5a3-4461-90ec-5d02c1d8c2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.334803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd922b-b5a3-4461-90ec-5d02c1d8c2a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:41.335111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd922b-b5a3-4461-90ec-5d02c1d8c2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:41.337566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd922b-b5a3-4461-90ec-5d02c1d8c2a1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:41.345716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 244/499] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:41.347055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b99d0-6414-4ec7-ab30-d701340ce04e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.347371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b99d0-6414-4ec7-ab30-d701340ce04e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.347629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b99d0-6414-4ec7-ab30-d701340ce04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.403741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:41.425639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37ed3c-c9dc-41bb-8cde-cf9a8226ed5e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.471325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f455bb-fbcb-4e23-aa66-0d745c27676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.471996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f455bb-fbcb-4e23-aa66-0d745c27676e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.473866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f455bb-fbcb-4e23-aa66-0d745c27676e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.474168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f455bb-fbcb-4e23-aa66-0d745c27676e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.487024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-268b99d0-6414-4ec7-ab30-d701340ce04e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:41.488510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-268b99d0-6414-4ec7-ab30-d701340ce04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:41.544381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f455bb-fbcb-4e23-aa66-0d745c27676e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:41.545869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f455bb-fbcb-4e23-aa66-0d745c27676e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:41.557959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 245/500] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:41.558805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b99d0-6414-4ec7-ab30-d701340ce04e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:41.630702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:41.631554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:41.656804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 256/501] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:41.658164 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934c0e9-af42-474f-bc7b-2d5c24e704cc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.658460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934c0e9-af42-474f-bc7b-2d5c24e704cc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.658749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934c0e9-af42-474f-bc7b-2d5c24e704cc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.660013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 257/502] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:41.663547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934c0e9-af42-474f-bc7b-2d5c24e704cc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:41.672409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b99d0-6414-4ec7-ab30-d701340ce04e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.705785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.706498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.706781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.707056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.714818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:41.716290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:41.732953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:41.739484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 246/503] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:41.740423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:41.765939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934c0e9-af42-474f-bc7b-2d5c24e704cc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.801014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.801814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.802100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.802379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.812827 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:41.814303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:41.831955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:41.832870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:41.833706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:41.833706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:41.833706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:41.833706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:41.833706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 258/504] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:41.833972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:41.857408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04491ef9-03c1-4c83-bb14-a285aaee89e9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.866419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:41.882575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:41.892943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe16933-109e-45b2-b367-fad2b07b0c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.893742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16933-109e-45b2-b367-fad2b07b0c40 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.894024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16933-109e-45b2-b367-fad2b07b0c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.894297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe16933-109e-45b2-b367-fad2b07b0c40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.905322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe16933-109e-45b2-b367-fad2b07b0c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ebe16933-109e-45b2-b367-fad2b07b0c40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:41.912975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:41.914000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:41.914868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 247/505] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:41.915758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2e97a-b909-403d-8fd3-f1dbe67a147d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.916051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2e97a-b909-403d-8fd3-f1dbe67a147d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.916350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2e97a-b909-403d-8fd3-f1dbe67a147d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.932698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d8c2e97a-b909-403d-8fd3-f1dbe67a147d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:41.933903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d8c2e97a-b909-403d-8fd3-f1dbe67a147d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:41.941497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b8207-ec90-4551-9da5-9488cc35228f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:41.976246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:41.976969 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:41.977346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:41.977678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:41.987691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:41.989180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.000428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 248/506] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:42.001271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2e97a-b909-403d-8fd3-f1dbe67a147d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:42.006544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:42.007544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:42.009027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:42.009027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:42.009027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:42.009027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:42.009027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 259/507] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:42.009463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.095914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:42.109513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2e97a-b909-403d-8fd3-f1dbe67a147d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.112587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584418a-d4e8-49be-873e-b88fa6fc7ec0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.135549 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:42.158118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.158900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.159178 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.159451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.159590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.160203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.160352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:42.160477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.160625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.160750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.161471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:42.161780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.162065 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.163146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.176367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 260/508] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:42.177211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.182147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 249/509] 10.222.0.32 () {56 vars in 1137 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:42.183055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.285607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926a79f-b407-4b9b-b310-e2899e702c97 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.287539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb5aad-b9fc-46da-9f7a-501bcea45f12 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.319579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.320282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.320575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.320859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.321606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:42.321917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.322938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.323294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.323600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.323877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.324148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.324892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:42.325195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.326542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.331419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 261/510] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:42.332245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.338626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 250/511] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:42.339301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.347046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:42.374207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:42.439796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cf4b9-5973-4e74-af4a-8dcf47c4983b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.439989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697d2e8-989d-42bd-b5b6-21f6677284eb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.474708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.475505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.475784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.476057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.482488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.483289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.483580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.483892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.490974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.492988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.497069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.498520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:42.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:42.512557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:42.513410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:42.513410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:42.513410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:42.513410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:42.513410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 262/512] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:42.513699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:42.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 30 11:50:42.515381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:42.516697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.516852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 251/513] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:42.516852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.578849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:42.612741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:42.620107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d24f6-6d3c-4a16-826c-b36b25fb7815 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.623468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad72a3d-23ca-43cf-8d9c-567358ad83f3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.655485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.656319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.656608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.656890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.658161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff64bc1-28a0-4269-96af-9ad63bbd169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.658839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64bc1-28a0-4269-96af-9ad63bbd169d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.659114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64bc1-28a0-4269-96af-9ad63bbd169d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.659465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64bc1-28a0-4269-96af-9ad63bbd169d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.668343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.669797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:42.686940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:42.687882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:42.688791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:42.688791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:42.688791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:42.688791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:42.688791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 252/514] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:42.689051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.732670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff64bc1-28a0-4269-96af-9ad63bbd169d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.734145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff64bc1-28a0-4269-96af-9ad63bbd169d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.791778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:42.793520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa1ce-9641-4374-a74b-70793d8cbe8e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.826719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.827512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.827801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.828076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.834599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 263/515] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:42.834867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.835375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff64bc1-28a0-4269-96af-9ad63bbd169d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.836681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:42.852780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:42.853722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:42.854597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:42.854597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:42.854597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:42.854597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:42.854597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:42.854597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:42.854597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:42.854597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:42.856466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 253/516] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:42.857159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.866035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:42.938677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff64bc1-28a0-4269-96af-9ad63bbd169d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.958228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e520679-cdae-4dfa-a32d-5a3ba118f64c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:42.973621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.974284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.974569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.974850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.975573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:42.975881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.977342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:42.988439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 264/517] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:42.989235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:42.994337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:42.995019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:42.995307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:42.995591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:42.996471 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:42.996788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:42.998219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.011691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:43.016647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 254/518] 10.222.0.32 () {56 vars in 1137 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:43.017725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:43.080890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:43.099544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc05a9-7f14-4668-9666-c3ac2b774349 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.119611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4d28d-a521-4275-bde5-7531ca74cb8f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.133714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:43.134387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:43.134672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:43.134952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:43.135676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:43.135991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:43.137413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.149833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 265/519] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:43.150653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:43.157625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:43.158425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:43.158713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:43.158992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:43.169452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:43.170899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:43.188832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:43.189673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:43.190761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:43.190761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:43.190761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:43.190761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:43.190761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 255/520] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:43.190761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:43.231688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:43.255231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c242d0b-b4bc-43a1-bfa7-4817e51345bb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.289972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c660a1-f304-421f-b49e-94c04e86d4c0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.293499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:43.294602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:43.294888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:43.295165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:43.295951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:43.296301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:43.297750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.307198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:43.316424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 266/521] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:43.317500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:43.330050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bfebf-e906-4ae0-adf7-9a738da1d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:43.330734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bfebf-e906-4ae0-adf7-9a738da1d5cf None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:43.331020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bfebf-e906-4ae0-adf7-9a738da1d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:43.331298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67bfebf-e906-4ae0-adf7-9a738da1d5cf None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:43.332080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67bfebf-e906-4ae0-adf7-9a738da1d5cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:43.333474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67bfebf-e906-4ae0-adf7-9a738da1d5cf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.354602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 256/522] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:50:43 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:50:43.355356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bfebf-e906-4ae0-adf7-9a738da1d5cf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:43.445340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d92c6-c327-4986-a7b0-99bda4cb1329 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.472864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67bfebf-e906-4ae0-adf7-9a738da1d5cf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.485876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716dd77-47e3-44dc-b6b2-956d60e1da8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:43.486563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716dd77-47e3-44dc-b6b2-956d60e1da8e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:43.486862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716dd77-47e3-44dc-b6b2-956d60e1da8e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:43.487132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716dd77-47e3-44dc-b6b2-956d60e1da8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:43.498608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:43.518927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:43.520069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:43.520406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:43.520691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:43.534236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:43.536681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:43.559469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:43.560483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 257/523] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:43.566940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:43.598356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716dd77-47e3-44dc-b6b2-956d60e1da8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:43.600271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716dd77-47e3-44dc-b6b2-956d60e1da8e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.609841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:43.698024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e49c14-1d0f-4cae-928a-d8edf8a4ed64 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.729220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 267/524] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:43.730668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716dd77-47e3-44dc-b6b2-956d60e1da8e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:43.753962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:43.754647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:43.759000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:43.759000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:43.759000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:43.759000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:43.759000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.774168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 258/525] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:43.779483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:43.794184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:43.879130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716dd77-47e3-44dc-b6b2-956d60e1da8e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.893193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:43.911339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f804a-6f52-44d6-904e-9a754c906612 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:43.927466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:43.928330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:43.928737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:43.929019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:43.930819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=70dce2f0284847ab8f07f55d700d0f60) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:43.931261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:43.932926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:43.945798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 268/526] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:43 2022] GET /identity/v3/role_assignments?user.id=70dce2f0284847ab8f07f55d700d0f60&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:43.945798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:44.014620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:44.015338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:44.015623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:44.015901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:44.024567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:44.025167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:44.027756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:44.063342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 259/527] 10.222.0.32 () {56 vars in 1139 bytes} [Mon May 30 11:50:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:44.063342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:44.131595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0887-9f1c-46ab-bc12-111bd6ad2b10 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:44.135227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:44.187186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:44.189046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:44.189334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:44.189619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:44.197258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde38653-65ed-4e38-8e7e-afd857792091 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:44.204958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:44.206450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:44.237665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:44.238762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:44.239989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:44.239989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:44.239989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:44.239989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:44.239989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 269/528] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:44.240523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:44.248094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:44.249089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:44.249531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:44.249911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:44.257930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:44.259680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:44.282895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:44.283908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:44.285023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 260/529] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:44.286069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:44.304393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:44.388877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9879c7-08f9-4289-a634-d85f1a24b361 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:44.423341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4587d20-c822-45d9-a3b4-4fdbe9646033 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:44.450121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103252d-0211-4dca-8e19-d082c1b31c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:44.451619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103252d-0211-4dca-8e19-d082c1b31c97 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:44.452378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103252d-0211-4dca-8e19-d082c1b31c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:44.452799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103252d-0211-4dca-8e19-d082c1b31c97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:44.467946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:44.468692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:44.468982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:44.469263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:44.470015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:44.470844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:44.472709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:44.489699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:44.490883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 261/530] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:44.492967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:44.553807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103252d-0211-4dca-8e19-d082c1b31c97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:44.557133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103252d-0211-4dca-8e19-d082c1b31c97 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:44.594247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:44.637174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe2000-06b0-48d9-81cd-761b31a4b2cb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:44.685421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:44.686393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:44.686680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:44.686954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:44.687755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:44.688069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:44.690379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:44.703119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 270/531] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:44.704005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103252d-0211-4dca-8e19-d082c1b31c97 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:44.715049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 262/532] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:44.715931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:44.788511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:44.837351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103252d-0211-4dca-8e19-d082c1b31c97 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:44.847059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63d3d0-021f-4921-a97f-24cd882dfa9d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:44.877086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:44.884188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:44.884999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:44.885303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:44.885570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:44.886038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:44.886855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:44.887137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:44.887421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:44.897045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:44.898673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:44.899175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:44.900339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:44.916666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:44.918358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 271/533] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:44.920049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:44.922071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:44.923664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:44.925496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:44.925496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:44.925496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:44.925496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:44.925496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 263/534] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:44.925496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:45.046028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4d742-1dd8-452f-a22b-b861a6ae7893 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:45.063830 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01206c46-7105-4405-a845-e503c0d9a54a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:45.084826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:45.085614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:45.086302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:45.086422 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:45.086708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:45.087911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:45.088260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:45.089808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:45.104086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 264/535] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:45.105190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:45.112818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:45.113768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:45.114787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:45.115272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:45.116330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=ad37bdce39f64836b0cd14613cabe91f) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:45.117287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:45.119469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:45.127920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 272/536] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:44 2022] GET /identity/v3/role_assignments?user.id=ad37bdce39f64836b0cd14613cabe91f&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:45.128956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:45.198096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:45.265063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f05f46-e924-4276-a59a-afa1b2c6c553 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:45.309648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:45.310655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:45.311131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:45.311611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:45.313626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=7c3c27e908e34977b248042f96c70dd1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:45.314318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:45.316301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:45.324342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 273/537] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:45 2022] GET /identity/v3/role_assignments?user.id=7c3c27e908e34977b248042f96c70dd1&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:45.360848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698c7fb-0633-42fc-bae0-2e4f661b7a52 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:45.420537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:45.421238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:45.421527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:45.421810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:45.422564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:45.422953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:45.424468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:45.438416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 265/538] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:45.446494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:45.504945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:45.596691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc3637-71eb-42cc-8424-481635aa405a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:45.638049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59411677-35a2-466a-96d5-73e3df3383de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:45.638861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59411677-35a2-466a-96d5-73e3df3383de None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:45.639145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59411677-35a2-466a-96d5-73e3df3383de None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:45.639425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59411677-35a2-466a-96d5-73e3df3383de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:45.646941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59411677-35a2-466a-96d5-73e3df3383de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:45.648673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59411677-35a2-466a-96d5-73e3df3383de None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-59411677-35a2-466a-96d5-73e3df3383de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:45.663617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:45.665543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:45.666453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 274/539] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:45.673348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:45.814747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eac58-0369-403a-97e8-ff58a6b18882 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:45.816648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:45.871752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:45.872378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:45.872671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:45.872956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:45.873761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:45.874078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:45.875521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:45.895730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 266/540] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:45.896053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642cb46-70dd-4724-b055-a5974727172e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:45.896513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642cb46-70dd-4724-b055-a5974727172e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:45.896696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642cb46-70dd-4724-b055-a5974727172e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:45.898335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 275/541] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:45.901942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:45.904217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ce5b5-6b55-43d3-abd7-4738de88cddc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:45.904519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ce5b5-6b55-43d3-abd7-4738de88cddc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:45.904801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ce5b5-6b55-43d3-abd7-4738de88cddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:45.922899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-163ce5b5-6b55-43d3-abd7-4738de88cddc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:45.924246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-163ce5b5-6b55-43d3-abd7-4738de88cddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:46.036209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 276/542] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:46.083177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565a69f-de66-40db-96fd-b58ade9420f5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:46.099577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ce5b5-6b55-43d3-abd7-4738de88cddc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:46.139226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:46.163143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:46.164218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:46.164676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:46.165188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:46.166336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=15ea38389eee4472a8917023643be9b1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:46.167011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:46.169566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:46.186660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 267/543] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:45 2022] GET /identity/v3/role_assignments?user.id=15ea38389eee4472a8917023643be9b1&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:46.238340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ce5b5-6b55-43d3-abd7-4738de88cddc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:46.286292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631aca5-cd22-4cd4-be02-5ce9b55ae280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:46.287116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631aca5-cd22-4cd4-be02-5ce9b55ae280 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:46.287402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631aca5-cd22-4cd4-be02-5ce9b55ae280 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:46.287680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631aca5-cd22-4cd4-be02-5ce9b55ae280 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:46.306722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631aca5-cd22-4cd4-be02-5ce9b55ae280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1631aca5-cd22-4cd4-be02-5ce9b55ae280 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:46.314924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:46.316725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:46.318290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:46.320601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 277/544] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:46.329353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:46.381918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:46.488763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793620a0-4392-4258-865a-79a222c52b03 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:46.539320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:46.540736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:46.541236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:46.541712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:46.542699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:46.543218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:46.544831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:46.548134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda8f03-60d2-4d70-aa5c-e259ff2f5251 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:46.548453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda8f03-60d2-4d70-aa5c-e259ff2f5251 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:46.548742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda8f03-60d2-4d70-aa5c-e259ff2f5251 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:46.550111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 278/545] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:50:46.555430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990fc8-9968-47ca-aabf-d44ec585cc04 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:46.555720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990fc8-9968-47ca-aabf-d44ec585cc04 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:46.556008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53990fc8-9968-47ca-aabf-d44ec585cc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:46.559197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 268/546] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:46.565136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:46.574770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-53990fc8-9968-47ca-aabf-d44ec585cc04 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:46.575973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-53990fc8-9968-47ca-aabf-d44ec585cc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:46.696992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 279/547] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:50:46.732524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:46.752101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4650f2e-9da8-45f6-9c2d-d966b0a622f5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:46.757090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53990fc8-9968-47ca-aabf-d44ec585cc04 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:46.800522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:46.811325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:46.812128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:46.812433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:46.812709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:46.829347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:46.830962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:46.850166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:46.851163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:46.852629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:46.852629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:46.852629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:46.852629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:46.852629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 269/548] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:46.858691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:46.944111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53990fc8-9968-47ca-aabf-d44ec585cc04 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:46.995171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6a65d-9ae1-4626-bc1c-eb346b49d475 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:47.011877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201df4d0-00d5-4e24-a251-c29cacc3e316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:47.013061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201df4d0-00d5-4e24-a251-c29cacc3e316 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:47.013546 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201df4d0-00d5-4e24-a251-c29cacc3e316 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:47.014135 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201df4d0-00d5-4e24-a251-c29cacc3e316 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:47.033999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:47.034976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:47.035271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:47.035561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:47.036325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:47.036643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:47.038759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:47.042199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201df4d0-00d5-4e24-a251-c29cacc3e316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-201df4d0-00d5-4e24-a251-c29cacc3e316 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:47.057296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:47.058738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:47.059726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:47.061190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:47.064345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 270/550] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:47.065050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 280/550] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:47.068462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:47.070375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201df4d0-00d5-4e24-a251-c29cacc3e316 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:47.224143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23f863-35b4-4c90-9123-cf5a191791b2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:47.268608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:47.268901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:47.269708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:47.269829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:47.270018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:47.270779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:47.271075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:47.272503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:47.282325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 271/551] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:47.287775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:47.300010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201df4d0-00d5-4e24-a251-c29cacc3e316 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:47.337797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:47.338993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:47.339500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:47.339993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:47.348466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:47.350292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:47.356975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:47.366170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:47.367887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 30 11:50:47.368985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:47.371706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 281/552] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:50:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:47.387032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:47.408546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390883d3-9f05-41e5-bc93-f06389eaca54 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:47.490694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:47.491500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:47.491779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:47.492347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:47.505991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:47.507673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:47.527830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:47.528758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:47.529706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:47.529706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:47.529706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:47.529706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:47.529706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 272/553] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:47.536185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:47.543628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad981a1-5009-4533-b5fa-9035334e167e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:47.614845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:47.615936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:47.616487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:47.617498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:47.618625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:47.619184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:47.620939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:47.684315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:47.691140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 282/554] 10.222.0.32 () {56 vars in 1135 bytes} [Mon May 30 11:50:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:47.697160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:47.713176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a723fb6-f74b-4cbc-b8d7-a5a2f88febb0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:47.735695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:47.780119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:47.780744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:47.781024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:47.781295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:47.782028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:47.782341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:47.785045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:47.811201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 273/555] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:47.818301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:47.875967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d2d07-dd74-4a5a-bf83-76a32b5f19e3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:47.945552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cff24-5525-4884-805e-5d8006b9f99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:47.946518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cff24-5525-4884-805e-5d8006b9f99e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:47.946884 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cff24-5525-4884-805e-5d8006b9f99e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:47.947163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cff24-5525-4884-805e-5d8006b9f99e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:47.954480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d4621-b0f3-481e-bdbb-b2e09921c903 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.005254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a980bd-dca7-49c0-ab00-af54a73ef61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:48.006069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a980bd-dca7-49c0-ab00-af54a73ef61b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.006357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a980bd-dca7-49c0-ab00-af54a73ef61b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.006638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a980bd-dca7-49c0-ab00-af54a73ef61b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.015733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a980bd-dca7-49c0-ab00-af54a73ef61b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:48.017949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a980bd-dca7-49c0-ab00-af54a73ef61b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-45a980bd-dca7-49c0-ab00-af54a73ef61b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:48.034497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:48.035436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:50:48.036852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:48.040287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 274/556] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:50:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:48.041966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251c660-fb8f-49ac-8c17-4db23784c94e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.042604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251c660-fb8f-49ac-8c17-4db23784c94e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.043197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251c660-fb8f-49ac-8c17-4db23784c94e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.048927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 275/557] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:48.049506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:48.050619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251c660-fb8f-49ac-8c17-4db23784c94e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.051695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cff24-5525-4884-805e-5d8006b9f99e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:48.053364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cff24-5525-4884-805e-5d8006b9f99e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:48.108543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:48.165595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 283/558] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 30 11:50:48.167091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b995b-fcf3-4492-9efb-c6dd3ef0db63 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.167383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b995b-fcf3-4492-9efb-c6dd3ef0db63 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.167665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b995b-fcf3-4492-9efb-c6dd3ef0db63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.175793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251c660-fb8f-49ac-8c17-4db23784c94e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.187804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4b7b995b-fcf3-4492-9efb-c6dd3ef0db63 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:48.189179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4b7b995b-fcf3-4492-9efb-c6dd3ef0db63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:48.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abdde2-dfef-448d-859c-23dbd4b62cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:48.214627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abdde2-dfef-448d-859c-23dbd4b62cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.214915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abdde2-dfef-448d-859c-23dbd4b62cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.215196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abdde2-dfef-448d-859c-23dbd4b62cf9 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.216317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abdde2-dfef-448d-859c-23dbd4b62cf9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:48.216637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abdde2-dfef-448d-859c-23dbd4b62cf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:48.218077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abdde2-dfef-448d-859c-23dbd4b62cf9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:48.241523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 276/559] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:48.241523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92713e5e-acb7-422a-bb39-866681514cd4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.241523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92713e5e-acb7-422a-bb39-866681514cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.241523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92713e5e-acb7-422a-bb39-866681514cd4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.241523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 277/560] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:48.241523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92713e5e-acb7-422a-bb39-866681514cd4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.269106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 284/561] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:48.281079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b995b-fcf3-4492-9efb-c6dd3ef0db63 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.318574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:48.344619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:48.355064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92713e5e-acb7-422a-bb39-866681514cd4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.393356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96ed5b-4352-424a-9fb6-2626d8ad2859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:48.394449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96ed5b-4352-424a-9fb6-2626d8ad2859 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.394873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96ed5b-4352-424a-9fb6-2626d8ad2859 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.395357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96ed5b-4352-424a-9fb6-2626d8ad2859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.400461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b995b-fcf3-4492-9efb-c6dd3ef0db63 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.407241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96ed5b-4352-424a-9fb6-2626d8ad2859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:48.408738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96ed5b-4352-424a-9fb6-2626d8ad2859 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-fa96ed5b-4352-424a-9fb6-2626d8ad2859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:48.427165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:48.428339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:48.429253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:48.429253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:48.429253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:48.429253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:48.429253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 278/562] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:48.429726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31ee8-e935-40e1-b670-b9f304ba342b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.430017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31ee8-e935-40e1-b670-b9f304ba342b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.430299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb31ee8-e935-40e1-b670-b9f304ba342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.444601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:48.445713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.446211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.446641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.447823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1cb31ee8-e935-40e1-b670-b9f304ba342b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:48.449068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1cb31ee8-e935-40e1-b670-b9f304ba342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:48.459211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:48.461929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:48.482229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:48.483363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:48.484492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:48.484492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:48.484492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:48.484492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:48.486127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 285/563] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:48.487098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.521950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 279/564] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:48.523683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb31ee8-e935-40e1-b670-b9f304ba342b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.592880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:48.613908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:48.614836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04c78-eff5-4fb1-8a2a-c2856225cd07 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.640973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb31ee8-e935-40e1-b670-b9f304ba342b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.652669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497a998-eb60-45de-921e-105e189815c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:48.653479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497a998-eb60-45de-921e-105e189815c8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.653767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497a998-eb60-45de-921e-105e189815c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.654070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497a998-eb60-45de-921e-105e189815c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.665198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497a998-eb60-45de-921e-105e189815c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c497a998-eb60-45de-921e-105e189815c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:48.672913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:48.673902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:48.674760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 286/565] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:48.675454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497a998-eb60-45de-921e-105e189815c8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.682970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:48.683653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.683945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.684253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.684991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:48.685304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:48.686692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:48.700678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 280/566] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:48.701922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.803427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497a998-eb60-45de-921e-105e189815c8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.810984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7fea-738f-4b6b-940f-58368dab0d71 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.840061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:48.840838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.841034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.841320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.842195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:48.842382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:48.843925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:48.847131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48315d6-1788-4d30-98e3-76b61746827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:48.847937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48315d6-1788-4d30-98e3-76b61746827d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:48.848261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48315d6-1788-4d30-98e3-76b61746827d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:48.848549 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48315d6-1788-4d30-98e3-76b61746827d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:48.850612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:48.860315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 287/567] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:48.861228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.864303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48315d6-1788-4d30-98e3-76b61746827d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:48.865935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d48315d6-1788-4d30-98e3-76b61746827d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:48.873706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:48.874648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:48.875489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:48.876119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 281/568] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:48.876742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48315d6-1788-4d30-98e3-76b61746827d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:48.973680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f967dd-3d89-4e28-93c6-36a34edb9073 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:48.994154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48315d6-1788-4d30-98e3-76b61746827d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.011082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.011456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.011747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.012057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.012789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:49.013098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.014606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.025155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 288/569] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:49.030355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.032692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.033503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.033792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.034075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.041409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.043227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:49.058782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:49.059927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:49.060910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 282/570] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:49.062395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.097160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:49.119414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:49.137267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cf242-63dd-441d-bd6e-8a1b0ee13db4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.164432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c50b8a-e443-40e7-9c43-4cdfee763d03 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.172521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.173348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.173639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.173923 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.181485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.182939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:49.197711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:49.198702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:49.199583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 289/571] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:49.200074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.204064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41486219-f021-4b21-a8bb-0ae0050b9984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.204761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486219-f021-4b21-a8bb-0ae0050b9984 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.205045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486219-f021-4b21-a8bb-0ae0050b9984 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.205320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486219-f021-4b21-a8bb-0ae0050b9984 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.206071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486219-f021-4b21-a8bb-0ae0050b9984 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:49.206385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486219-f021-4b21-a8bb-0ae0050b9984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.207752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486219-f021-4b21-a8bb-0ae0050b9984 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.219167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 283/572] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:49.219167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd34a-b7ef-4f39-9061-ff9da64b23d3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.219167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd34a-b7ef-4f39-9061-ff9da64b23d3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.219167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd34a-b7ef-4f39-9061-ff9da64b23d3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.219167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 284/573] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:49.219790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd34a-b7ef-4f39-9061-ff9da64b23d3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.306487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6b66d-e948-4aa5-b7c6-b2ecc9b00814 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.325410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd34a-b7ef-4f39-9061-ff9da64b23d3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.330410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:49.341766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:49.342045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.342852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.343139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.343476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.355057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.356516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.359998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.360685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.360964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.361240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.362023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:49.362328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.363731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:49.374346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:49.375423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:49.376644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:49.376644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:49.376644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:49.376644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:49.376644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 290/574] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:49.376939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 285/575] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:49.377247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.378015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.479943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2784d1-cd32-4a60-8972-27084b462018 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.480211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a6393-e62d-40e7-990d-174cc720c05a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.514987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78da1e5-bbb8-42eb-94b4-874b555ea167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.515647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78da1e5-bbb8-42eb-94b4-874b555ea167 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.515932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78da1e5-bbb8-42eb-94b4-874b555ea167 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.516242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78da1e5-bbb8-42eb-94b4-874b555ea167 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.517278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78da1e5-bbb8-42eb-94b4-874b555ea167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:49.517492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.517611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78da1e5-bbb8-42eb-94b4-874b555ea167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.518287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.518572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.518853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.519051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78da1e5-bbb8-42eb-94b4-874b555ea167 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.529666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.531097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.532196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 291/576] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:49.533521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da6f4-41bc-41ce-a22e-bfefce8d8fee None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.533811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da6f4-41bc-41ce-a22e-bfefce8d8fee None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.534093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8da6f4-41bc-41ce-a22e-bfefce8d8fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.549990 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8c8da6f4-41bc-41ce-a22e-bfefce8d8fee None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:49.551184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8c8da6f4-41bc-41ce-a22e-bfefce8d8fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:49.553856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:49.554934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:49.556134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:49.556134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:49.556134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:49.556134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:49.556134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 286/577] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:49.556586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.584086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:49.585353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:49.627522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 292/578] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:49.628395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8da6f4-41bc-41ce-a22e-bfefce8d8fee None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.667758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cfe18-bb63-4a0b-8051-7a25ce083e9d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.703664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.704362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.704647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.704924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.705649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:49.705960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.707333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.719547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 287/579] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:49.720399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.745974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8da6f4-41bc-41ce-a22e-bfefce8d8fee None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.782623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.783587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.783873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.784150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.784996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=e0cb45c1eb6040548634022673a51e56) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:49.785307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.786757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.792650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 293/580] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/role_assignments?user.id=e0cb45c1eb6040548634022673a51e56&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:49.793459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.828882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:49.830730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:49.848113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68118fa6-93a7-4cde-b91d-e5df4bad96b2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.890964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc47c-9cfd-4f85-bb2d-9da87d4096a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.891771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc47c-9cfd-4f85-bb2d-9da87d4096a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.892445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc47c-9cfd-4f85-bb2d-9da87d4096a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.892731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc47c-9cfd-4f85-bb2d-9da87d4096a8 None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/70dce2f0284847ab8f07f55d700d0f60/roles/126791116ac8403986eead13c8c6dd07` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.910139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc47c-9cfd-4f85-bb2d-9da87d4096a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=70dce2f0284847ab8f07f55d700d0f60, role_id=126791116ac8403986eead13c8c6dd07)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.911890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc47c-9cfd-4f85-bb2d-9da87d4096a8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.914339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f2e27-c97a-4787-83de-697869d57505 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:49.932281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 288/581] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:49 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/70dce2f0284847ab8f07f55d700d0f60/roles/126791116ac8403986eead13c8c6dd07 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:49.933363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc47c-9cfd-4f85-bb2d-9da87d4096a8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:49.966637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:49.967337 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:49.967627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:49.967948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:49.968820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:49.969940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:49.971780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:49.990284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 294/582] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:49.994485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.085425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc47c-9cfd-4f85-bb2d-9da87d4096a8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.136498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ff145-beaa-4ca4-9497-1d194a30da72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.138616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff145-beaa-4ca4-9497-1d194a30da72 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.139579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff145-beaa-4ca4-9497-1d194a30da72 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.140413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff145-beaa-4ca4-9497-1d194a30da72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.152046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:50.156764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ff145-beaa-4ca4-9497-1d194a30da72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:50.159983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b70ff145-beaa-4ca4-9497-1d194a30da72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:50.168277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:50.169297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:50.170151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 289/583] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:50.170856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ff145-beaa-4ca4-9497-1d194a30da72 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.176189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822e6d5-02cc-45d8-bb84-20a47450312f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.217697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.218999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.219559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.220039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.229825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:50.231816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:50.250014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:50.251002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 30 11:50:50.251992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:50.255735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 295/584] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:50.256924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.294524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ff145-beaa-4ca4-9497-1d194a30da72 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.347412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.348264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.348551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.348826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.360330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:50.362272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:50.371160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b018c-5c74-4251-a99d-d69dda376d4b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:50.380775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:50.381762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:50.383257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 290/585] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:50.384354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.418410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.419913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.420735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.421547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.423142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:50.424324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:50.426938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:50.445498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 296/586] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:50.447442 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.455508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:50.466841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:50.515242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba12bdb-4d9e-48e2-ac39-ef4e23e94176 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.565235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.566174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.566465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.566743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.567597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:50.568176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:50.570568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:50.577913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b0763-7f74-4665-8606-1944d85d0f95 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.596813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 291/587] 10.222.0.32 () {56 vars in 1137 bytes} [Mon May 30 11:50:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:50.597739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.625562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.626641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.627136 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.627613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.628704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:50.629269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:50.631001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:50.651402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 297/588] 10.222.0.32 () {56 vars in 1137 bytes} [Mon May 30 11:50:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:50.652621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.722770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108bd9f-3de6-4b36-b799-564a54f67f4a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.740174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:50.740377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:50.763376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.764424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.764821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.765229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.777547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:50.779254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:50.781014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937ed73-de5d-4d7c-90c6-b630ed52d03f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:50.798851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:50.799953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:50.800997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:50.800997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:50.800997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:50.800997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:50.807048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 292/589] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:50.808148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.825447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.826788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.827247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.827721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.840879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:50.842567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:50.861989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:50.863014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:50.863875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:50.863875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:50.863875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:50.863875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:50.868012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 298/590] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:50.868988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:50.940934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df06f76-d379-4a3c-9b09-94dda7f97831 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.983537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:50.984405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:50.984859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:50.985023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:50.996181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38af251-41cf-45c5-ba15-90631325774f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:50.996671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:50.997835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:51.019092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:51.020023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:51.020929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:51.020929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:51.020929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:51.020929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:51.023469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 293/591] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:51.024801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.025556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:51.040887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.042145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.042507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.042869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.043966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:51.044410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.046608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.053619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:51.068317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 299/592] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:51.069490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.152770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dbcb4-c517-4855-a2b5-59f0f246d651 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.192372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.193347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.193806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.194280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.194492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c455e2b-0d7f-4d9c-a319-26168f34c5f6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.196175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:51.196591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.198005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.214622 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 294/593] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:51.215561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.231743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.232450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.232737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.233017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.233799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:51.234056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.235439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.249606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 300/594] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:51.250476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.296601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:51.301851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:51.324474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208089b-748b-4a2c-ab5b-9a8c3d5d49ec None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.355097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515f1d7-9382-4edb-a60c-af5a0e69d62e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.360896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.361915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.362345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.362763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.372085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.373750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:51.389487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:51.390469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 30 11:50:51.391413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:51.393388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.394192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.394480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.394760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.400471 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 295/595] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:51.400471 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.411390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.413202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:51.429873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:51.430892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:51.431808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:51.431808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:51.431808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:51.431808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:51.431808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:51.431808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:51.431808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:51.431808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:51.438862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 301/596] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:51.439971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.512911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf3cde-c06c-484b-96b5-aa9d747fdb30 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.549171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.549974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.550261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.550542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.558157 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.559150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196c1a9-ff15-487b-8be9-22d3e8ec2b4b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.559612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.561147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:51.562389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:51.583367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:51.585125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:51.587485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 296/597] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:51.592193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.607992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.608857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.609384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.609695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.610789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:51.611134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.612780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.638943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 302/598] 10.222.0.32 () {56 vars in 1139 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:51.640035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.704542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420efe44-6225-431b-9229-646e931b4fa0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.743631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.744618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.745054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.745570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.746637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:51.747097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.748819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.752802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfb2e6-24c8-48e8-9cf4-cf5637679d8e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.765260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 297/599] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:51.766468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.818246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.820186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.821140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.821764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.824331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:51.825019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.828700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.849417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 303/600] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:51.852034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:51.855376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:51.885406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:51.904436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c7c91-fc2f-4481-a9be-cc3973fa680d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.941743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:51.942861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:51.943300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:51.943681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:51.955896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:51.957680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:51.978110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:51.979273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:51.980400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:51.980400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:51.980400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:51.980400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:51.985699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bf125-2786-4407-839b-f8c8b6f17dc1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:51.986379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 298/601] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:51.987355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:52.033227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1286-6cf3-4035-af7c-fc2278b17f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:52.035020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1286-6cf3-4035-af7c-fc2278b17f0d None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.035569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1286-6cf3-4035-af7c-fc2278b17f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.036065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1286-6cf3-4035-af7c-fc2278b17f0d None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/ad37bdce39f64836b0cd14613cabe91f/roles/126791116ac8403986eead13c8c6dd07` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.053710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c1286-6cf3-4035-af7c-fc2278b17f0d None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=ad37bdce39f64836b0cd14613cabe91f, role_id=126791116ac8403986eead13c8c6dd07)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:52.056263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c1286-6cf3-4035-af7c-fc2278b17f0d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:52.078575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 304/602] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:51 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/ad37bdce39f64836b0cd14613cabe91f/roles/126791116ac8403986eead13c8c6dd07 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:52.078575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1286-6cf3-4035-af7c-fc2278b17f0d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:52.140238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8415ef8-22f6-4307-a88b-3a0c4f449946 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:52.162740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:52.192253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad02a23-4f30-4897-9a70-765aec405857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:52.193171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02a23-4f30-4897-9a70-765aec405857 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.193517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02a23-4f30-4897-9a70-765aec405857 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.193803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad02a23-4f30-4897-9a70-765aec405857 None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/7c3c27e908e34977b248042f96c70dd1/roles/126791116ac8403986eead13c8c6dd07` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.199795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:52.211602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad02a23-4f30-4897-9a70-765aec405857 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=7c3c27e908e34977b248042f96c70dd1, role_id=126791116ac8403986eead13c8c6dd07)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:52.213148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad02a23-4f30-4897-9a70-765aec405857 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:52.231119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 299/603] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:51 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/7c3c27e908e34977b248042f96c70dd1/roles/126791116ac8403986eead13c8c6dd07 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:52.232626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591d3bf-9bcc-4d52-a837-561be4ad3f3f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.233037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591d3bf-9bcc-4d52-a837-561be4ad3f3f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.233322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591d3bf-9bcc-4d52-a837-561be4ad3f3f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.235242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 300/604] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:52.241424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3fca1-7eed-4022-9c01-e23f51307e6d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.241832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3fca1-7eed-4022-9c01-e23f51307e6d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.242284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3fca1-7eed-4022-9c01-e23f51307e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.252010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1286-6cf3-4035-af7c-fc2278b17f0d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:52.270977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-83a3fca1-7eed-4022-9c01-e23f51307e6d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:52.273453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-83a3fca1-7eed-4022-9c01-e23f51307e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:52.297857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:52.298177 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.298401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.298688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.299444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:52.299759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:52.301412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:52.318433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 305/605] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:52.326101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:52.386503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 301/606] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:52.414713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3fca1-7eed-4022-9c01-e23f51307e6d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:52.472951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19429108-1e46-4aaf-aae2-5588eb9230ad None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:52.519392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:52.526549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d59eb8-9a7e-467a-87ea-76ad3c18937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:52.527962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d59eb8-9a7e-467a-87ea-76ad3c18937b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.528767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d59eb8-9a7e-467a-87ea-76ad3c18937b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.529224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d59eb8-9a7e-467a-87ea-76ad3c18937b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.538368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d59eb8-9a7e-467a-87ea-76ad3c18937b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:52.542193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:52.542828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d59eb8-9a7e-467a-87ea-76ad3c18937b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:52.554323 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3fca1-7eed-4022-9c01-e23f51307e6d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-71d59eb8-9a7e-467a-87ea-76ad3c18937b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:52.564077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:52.565007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:52.565890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 306/607] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:52.567131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc3425-6586-43d8-a605-9d44cd985afc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.572568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc3425-6586-43d8-a605-9d44cd985afc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.572568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc3425-6586-43d8-a605-9d44cd985afc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.572568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 307/608] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:52.577112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc3425-6586-43d8-a605-9d44cd985afc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:52.611462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40384cf9-b803-4adf-9ce0-f3806f63aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:52.612952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40384cf9-b803-4adf-9ce0-f3806f63aaab None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.613864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40384cf9-b803-4adf-9ce0-f3806f63aaab None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.614609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40384cf9-b803-4adf-9ce0-f3806f63aaab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.632141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40384cf9-b803-4adf-9ce0-f3806f63aaab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-40384cf9-b803-4adf-9ce0-f3806f63aaab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:52.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:52.645203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:52.646107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:52.649691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 302/609] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:50:52.651831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeee0d3-75f5-4c7b-a118-f09cd902cc34 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.652418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeee0d3-75f5-4c7b-a118-f09cd902cc34 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.652846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeee0d3-75f5-4c7b-a118-f09cd902cc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.675226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5aeee0d3-75f5-4c7b-a118-f09cd902cc34 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:52.676911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5aeee0d3-75f5-4c7b-a118-f09cd902cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:52.704832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc3425-6586-43d8-a605-9d44cd985afc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:52.743283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:52.743957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.744281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.744557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.745394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:52.745707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:52.747353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:52.765555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 308/610] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:52.766487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:52.767888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 303/611] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:52.769011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeee0d3-75f5-4c7b-a118-f09cd902cc34 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:52.833022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:52.888836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:52.902636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839e48-2ebf-47d8-8914-605552121b8c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:52.918714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeee0d3-75f5-4c7b-a118-f09cd902cc34 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:52.940502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:52.941355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.941732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.942103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.942962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:52.943366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:52.944916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:52.953954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 309/612] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:52.954950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:52.966459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c431ed1-c975-4a89-8ef5-f643684058e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:52.968090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c431ed1-c975-4a89-8ef5-f643684058e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:52.968560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c431ed1-c975-4a89-8ef5-f643684058e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:52.968988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c431ed1-c975-4a89-8ef5-f643684058e8 None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/15ea38389eee4472a8917023643be9b1/roles/126791116ac8403986eead13c8c6dd07` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:52.991199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c431ed1-c975-4a89-8ef5-f643684058e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=15ea38389eee4472a8917023643be9b1, role_id=126791116ac8403986eead13c8c6dd07)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:52.993625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c431ed1-c975-4a89-8ef5-f643684058e8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:53.016397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 304/613] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:52 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/15ea38389eee4472a8917023643be9b1/roles/126791116ac8403986eead13c8c6dd07 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:53.017791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c431ed1-c975-4a89-8ef5-f643684058e8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:53.075792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c675aa9-0604-4ba9-9024-e13b5a00beb8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:53.091581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:53.156488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98918d7-4191-4c21-8fcc-b59c8b3651d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:53.157306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98918d7-4191-4c21-8fcc-b59c8b3651d9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:53.157590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98918d7-4191-4c21-8fcc-b59c8b3651d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:53.157869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98918d7-4191-4c21-8fcc-b59c8b3651d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:53.187020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c431ed1-c975-4a89-8ef5-f643684058e8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:53.188936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98918d7-4191-4c21-8fcc-b59c8b3651d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a98918d7-4191-4c21-8fcc-b59c8b3651d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:53.198266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:53.199289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:53.200129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 310/614] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:53.207334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98918d7-4191-4c21-8fcc-b59c8b3651d9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:53.238759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:53.248766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:53.249600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:53.249884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:53.250159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:53.279950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:53.281507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:53.318431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:53.319373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:53.321040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:53.321040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:53.321040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:53.321040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:53.321040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 305/615] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:53.326092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:53.398649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98918d7-4191-4c21-8fcc-b59c8b3651d9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:53.468897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:53.469665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:53.469958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:53.470236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:53.471817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:53.472140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:53.473623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:53.480552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca770308-3fe8-4503-9859-c642893af41f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:53.495902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 311/616] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:53.501777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:53.553711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:53.556979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:53.557265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:53.557543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:53.558284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:53.558606 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:53.558936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:53.560185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:53.586946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 306/617] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:53.591977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:53.689128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:53.698902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc83423-473a-4761-8e6b-e1fc14a4aed7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:53.760458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:53.761438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:53.761729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:53.762009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:53.785755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:53.788259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:53.801171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a39c05-1a3b-455f-9ae4-45aa6cb7ab35 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:53.833914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:53.834860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:53.835793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:53.835793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:53.835793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:53.835793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:53.840191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 312/618] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:53.842079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:53.874617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:53.875429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:53.875714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:53.875988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:53.893199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:53.897784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:53.928541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:53.931532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:53.933833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 307/619] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:50:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:53.940330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:53.977834 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:54.011703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974ebbb-ad3e-4be2-9fb5-4db2b6c4e35f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:54.070426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176a4d5-243f-4d03-bcc7-ccb80d4ed58a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:54.073408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:54.095056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:54.095744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.096004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.096313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.097041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:54.097350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:54.098733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:54.124710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 313/620] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:50:54.126455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:54.127326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.127757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.128216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.129833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:54.130989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:54.131852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:54.135765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:54.169007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 308/621] 10.222.0.32 () {56 vars in 1135 bytes} [Mon May 30 11:50:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:54.181236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:54.317398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7880-d3ee-4374-9e83-ca83de94c006 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:54.372592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0e690-a705-42ff-83bd-191c16e40794 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:54.378388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:54.379218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.379508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.380865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.397488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:54.399594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:54.419669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:54.420704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:50:54.421644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:54.433276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 314/622] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:50:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:54.437464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:54.449963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:54.450809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.451097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.451379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.458843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:54.474058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:54.475529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:54.503706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:54.504718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:54.505614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:54.505614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:54.505614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:54.505614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:54.518399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 309/623] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:54.520710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:54.553690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:54.623639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7dd35-f9ee-47a0-b33e-cd2d8fed83a4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:54.686308 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eabb8-6e1b-418c-8161-e4ac52eee1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:54.690151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eabb8-6e1b-418c-8161-e4ac52eee1ed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.690630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eabb8-6e1b-418c-8161-e4ac52eee1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.691666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eabb8-6e1b-418c-8161-e4ac52eee1ed None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.692706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eabb8-6e1b-418c-8161-e4ac52eee1ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:54.694543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eabb8-6e1b-418c-8161-e4ac52eee1ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:54.696727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eabb8-6e1b-418c-8161-e4ac52eee1ed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:54.711567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a263a-3d78-45f0-87a9-4e3cc412c0c1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:54.735322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 315/624] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:50:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:54.741644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01bce-492e-4334-940d-3b6cca6387d4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.741945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01bce-492e-4334-940d-3b6cca6387d4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.742364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01bce-492e-4334-940d-3b6cca6387d4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.744121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 316/625] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:54.744949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b01bce-492e-4334-940d-3b6cca6387d4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:54.764506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc276c01-9b04-4a37-a1f4-1e0546921759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:54.765183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276c01-9b04-4a37-a1f4-1e0546921759 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.765466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276c01-9b04-4a37-a1f4-1e0546921759 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.765743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276c01-9b04-4a37-a1f4-1e0546921759 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.766465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc276c01-9b04-4a37-a1f4-1e0546921759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:54.766773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc276c01-9b04-4a37-a1f4-1e0546921759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:54.768716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc276c01-9b04-4a37-a1f4-1e0546921759 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:54.781964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 310/626] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:54.786963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561dc10-4de8-40c7-932d-e1c90a98858d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.786963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561dc10-4de8-40c7-932d-e1c90a98858d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.786963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561dc10-4de8-40c7-932d-e1c90a98858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.804840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4561dc10-4de8-40c7-932d-e1c90a98858d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:54.806217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4561dc10-4de8-40c7-932d-e1c90a98858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:54.827281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:54.879418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b01bce-492e-4334-940d-3b6cca6387d4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:54.880199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 311/627] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:54.881054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561dc10-4de8-40c7-932d-e1c90a98858d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:54.884907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:54.924109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23fe9b-afbc-4bb0-a09b-af38301417b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:54.925046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23fe9b-afbc-4bb0-a09b-af38301417b2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:54.925631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23fe9b-afbc-4bb0-a09b-af38301417b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:54.925921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23fe9b-afbc-4bb0-a09b-af38301417b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:54.988540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561dc10-4de8-40c7-932d-e1c90a98858d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:54.998892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23fe9b-afbc-4bb0-a09b-af38301417b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:55.000864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23fe9b-afbc-4bb0-a09b-af38301417b2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:55.031037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed60341-e450-4a01-8b97-5f13a06d2c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.032108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60341-e450-4a01-8b97-5f13a06d2c36 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.032420 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60341-e450-4a01-8b97-5f13a06d2c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.032695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed60341-e450-4a01-8b97-5f13a06d2c36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.040518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed60341-e450-4a01-8b97-5f13a06d2c36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:55.042077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed60341-e450-4a01-8b97-5f13a06d2c36 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2ed60341-e450-4a01-8b97-5f13a06d2c36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:55.056453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:55.057395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:55.058368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 312/628] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:55.059186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c4c61-4f81-4909-b90a-aca863f4199b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.059186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c4c61-4f81-4909-b90a-aca863f4199b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.059444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c4c61-4f81-4909-b90a-aca863f4199b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.060909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 313/629] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:55.061445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c4c61-4f81-4909-b90a-aca863f4199b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.080530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:55.123756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 317/630] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:55.124929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381c3cc-1ecf-4f52-8ac5-09835bab6a54 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.125227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381c3cc-1ecf-4f52-8ac5-09835bab6a54 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.125509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381c3cc-1ecf-4f52-8ac5-09835bab6a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.139524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:55.143825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f381c3cc-1ecf-4f52-8ac5-09835bab6a54 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:55.145072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f381c3cc-1ecf-4f52-8ac5-09835bab6a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:55.188513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c4c61-4f81-4909-b90a-aca863f4199b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.220899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 318/631] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:55.221649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381c3cc-1ecf-4f52-8ac5-09835bab6a54 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.235751 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5764a-8eef-4024-bedf-f6d980128687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.237152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5764a-8eef-4024-bedf-f6d980128687 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.237648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5764a-8eef-4024-bedf-f6d980128687 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.238150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5764a-8eef-4024-bedf-f6d980128687 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.251444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5764a-8eef-4024-bedf-f6d980128687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-44b5764a-8eef-4024-bedf-f6d980128687 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:55.259733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:55.260791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:55.261649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:55.265766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 314/632] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:55.266871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5764a-8eef-4024-bedf-f6d980128687 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.331087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:55.339954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381c3cc-1ecf-4f52-8ac5-09835bab6a54 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.377361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.378728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.379196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.379858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.381003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:55.381771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:55.384233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:55.388059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5764a-8eef-4024-bedf-f6d980128687 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.402064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 319/633] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:55.403704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.414874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:55.427244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.428043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.428352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.428631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.440489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:55.441939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:55.463207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:55.464204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:55.465112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:55.465112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:55.465112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:55.465112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:55.465112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 315/634] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:55.465841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.521694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e3343-171f-462a-8e4d-5ddb60a05971 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.561660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be0509-35cd-4864-9861-689ccf9d1456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.562462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be0509-35cd-4864-9861-689ccf9d1456 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.562758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be0509-35cd-4864-9861-689ccf9d1456 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.563042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be0509-35cd-4864-9861-689ccf9d1456 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.576249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be0509-35cd-4864-9861-689ccf9d1456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-12be0509-35cd-4864-9861-689ccf9d1456 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:55.584721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:55.586234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:55.587092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 320/635] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:55.587782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be0509-35cd-4864-9861-689ccf9d1456 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.601610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ab856-b012-4a63-9799-00f039103288 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.606178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:55.638835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.639510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.639797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.640072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.640856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:55.641164 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:55.642795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:55.651517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 316/636] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:55.652429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.693174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:55.698906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be0509-35cd-4864-9861-689ccf9d1456 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.735658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0475c5-f58a-456e-9cbd-56e4f3fd9592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.736586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0475c5-f58a-456e-9cbd-56e4f3fd9592 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.736875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0475c5-f58a-456e-9cbd-56e4f3fd9592 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.737156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0475c5-f58a-456e-9cbd-56e4f3fd9592 None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/e0cb45c1eb6040548634022673a51e56/roles/126791116ac8403986eead13c8c6dd07` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.752538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0475c5-f58a-456e-9cbd-56e4f3fd9592 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=e0cb45c1eb6040548634022673a51e56, role_id=126791116ac8403986eead13c8c6dd07)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:55.754015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0475c5-f58a-456e-9cbd-56e4f3fd9592 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:55.755183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f5bc2-1e4b-446f-b937-76837a55d3cd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.767715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 321/637] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:50:55 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/e0cb45c1eb6040548634022673a51e56/roles/126791116ac8403986eead13c8c6dd07 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:50:55.768586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0475c5-f58a-456e-9cbd-56e4f3fd9592 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.799341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b0c21-bff8-49b0-ada0-568ed8d381f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.800123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b0c21-bff8-49b0-ada0-568ed8d381f8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.800443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b0c21-bff8-49b0-ada0-568ed8d381f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.800730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b0c21-bff8-49b0-ada0-568ed8d381f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.801479 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b0c21-bff8-49b0-ada0-568ed8d381f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:55.801912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b0c21-bff8-49b0-ada0-568ed8d381f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:55.803789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b0c21-bff8-49b0-ada0-568ed8d381f8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:55.817521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 317/638] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:55.818893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183ceae-9e7d-4865-ab29-f7313d74041a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.819184 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183ceae-9e7d-4865-ab29-f7313d74041a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.819461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183ceae-9e7d-4865-ab29-f7313d74041a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.821298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 318/639] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:55.822842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183ceae-9e7d-4865-ab29-f7313d74041a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:55.858514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:55.915476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0475c5-f58a-456e-9cbd-56e4f3fd9592 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.950829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:55.955816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183ceae-9e7d-4865-ab29-f7313d74041a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:55.957551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:55.958439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:55.959610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:55.959916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:55.961110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:55.961600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:55.963022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:55.975636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 322/640] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:55.976662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.003503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb58c22-b393-429a-865b-16ed13396268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.004439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58c22-b393-429a-865b-16ed13396268 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.004784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58c22-b393-429a-865b-16ed13396268 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.005126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb58c22-b393-429a-865b-16ed13396268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.016524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb58c22-b393-429a-865b-16ed13396268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.018235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb58c22-b393-429a-865b-16ed13396268 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1fb58c22-b393-429a-865b-16ed13396268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:56.037698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:56.038752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:56.039683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:56.039683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:56.039683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:56.039683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:56.039683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 319/641] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:56.040438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa8fb6-b1c1-4d7a-af8d-da1eb69dd42a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.040723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa8fb6-b1c1-4d7a-af8d-da1eb69dd42a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.040996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa8fb6-b1c1-4d7a-af8d-da1eb69dd42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.061202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f5aa8fb6-b1c1-4d7a-af8d-da1eb69dd42a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:56.062379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f5aa8fb6-b1c1-4d7a-af8d-da1eb69dd42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:56.090293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483cd43-57b7-4253-89a6-b51de47d91b3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.120456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:56.132258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.133116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.133371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.133651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.143650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.143650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.144275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 320/642] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:56.145537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa8fb6-b1c1-4d7a-af8d-da1eb69dd42a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:56.169010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:56.169988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:56.170870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 323/643] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:56.171345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.254002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:56.277832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa8fb6-b1c1-4d7a-af8d-da1eb69dd42a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.283086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2d6c3-2c36-4433-94ad-1acc294ff797 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.317400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.318065 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.318348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.318626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.327138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.327813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.328104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.328418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.329155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:56.329470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.330778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.330889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.332569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.342858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 324/644] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:56.343517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:56.357762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:56.358716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:56.364747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:56.364747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:56.364747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:56.364747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:56.364747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 321/645] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:56.364747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.396402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:56.457068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192de326-655a-4664-90c2-213b8e108c7b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.473864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4280ab4-f307-4710-8f12-36f4cb0f0559 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.495509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047c36e-7d9a-4010-83d6-a483574081c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.496355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047c36e-7d9a-4010-83d6-a483574081c8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.496643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047c36e-7d9a-4010-83d6-a483574081c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.496930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047c36e-7d9a-4010-83d6-a483574081c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.506931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:56.508332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047c36e-7d9a-4010-83d6-a483574081c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:56.513315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.513983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.514271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.514553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.515348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:56.515663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a047c36e-7d9a-4010-83d6-a483574081c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:56.516061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:56.517322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:56.518189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:56.518826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.518895 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 325/646] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:56.518895 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047c36e-7d9a-4010-83d6-a483574081c8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.535900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 322/647] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:56.536622 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.645947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047c36e-7d9a-4010-83d6-a483574081c8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.646320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb63dc6-040d-4ebd-a78a-40e3db2b6be4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.655240 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:56.682953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.683628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.683915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.684223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.684956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:56.685284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.686638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.691779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.692802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.693250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.694216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.703990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 323/648] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:56.703990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.704253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.704789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:56.720460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:56.721387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 30 11:50:56.722331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:56.726740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 326/649] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:56.727644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.763559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:56.818912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b2835-1c1d-4156-a660-052ecf4dc44c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.835024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e64c-e861-43a2-9fee-33b13ece11f7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:56.853789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.854583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.854927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.855268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.856086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:56.856588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.857943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.870275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 324/650] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:56.870275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.878221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:56.879027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:56.879380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:56.879845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:56.880732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=70dce2f0284847ab8f07f55d700d0f60) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:56.881056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:56.883153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:56.900196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 327/651] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/role_assignments?user.id=70dce2f0284847ab8f07f55d700d0f60&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:56.900196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:56.936725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:57.005119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243aade-a9c0-421c-81a1-32e1f9681ae7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:57.021478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:57.031775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3143f4-8c8e-47ff-a7b3-7a21d9f4cac5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:57.046994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:57.048076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:57.049003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:57.049282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:57.061841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:57.064224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:57.072736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:57.073534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:57.073897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:57.074251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:57.075196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:57.075586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:57.077168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:57.081338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:57.082303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:57.083235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 325/652] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:57.083973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:57.099100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 328/653] 10.222.0.32 () {56 vars in 1137 bytes} [Mon May 30 11:50:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:57.100196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:57.203829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:57.206638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c92d1f-3f28-47ef-a44f-d9c9d5909b82 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:57.222478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc7f8d-986f-491b-b557-f163e2c1e46c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:57.255237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:57.255979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:57.256296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:57.256582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:57.262489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:57.263174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:57.263466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:57.263747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:57.264635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:57.264950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:57.266408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:57.268767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:57.270404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:57.285676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 329/654] 10.222.0.32 () {56 vars in 1137 bytes} [Mon May 30 11:50:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:57.286540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:57.295210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:57.296129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:57.297072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:57.297072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:57.297072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:57.297072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:57.298878 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 326/655] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:57.299839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:57.336626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:57.409913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb3e4f-cb11-4b72-8036-aca3c19611d2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:57.428777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87660845-8bad-4c8d-8172-1a96ecec649f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:57.446218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d42724-6573-4d50-9067-a04d5fc4fda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:57.446889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d42724-6573-4d50-9067-a04d5fc4fda3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:57.447175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d42724-6573-4d50-9067-a04d5fc4fda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:57.448715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d42724-6573-4d50-9067-a04d5fc4fda3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:57.475527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:57.481740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c5292-732c-4076-b2de-765287c998fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:57.482961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c5292-732c-4076-b2de-765287c998fe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:57.483397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c5292-732c-4076-b2de-765287c998fe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:57.483850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c5292-732c-4076-b2de-765287c998fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:57.548494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d42724-6573-4d50-9067-a04d5fc4fda3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:57.550233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d42724-6573-4d50-9067-a04d5fc4fda3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:57.593823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c5292-732c-4076-b2de-765287c998fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:57.595761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c5292-732c-4076-b2de-765287c998fe None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:57.639167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:57.691715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 330/656] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:57.692772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d42724-6573-4d50-9067-a04d5fc4fda3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:57.746384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 327/657] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:57.749199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c5292-732c-4076-b2de-765287c998fe None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:57.755621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:57.826941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d42724-6573-4d50-9067-a04d5fc4fda3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:57.884848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:57.885553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:57.885841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:57.886123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:57.886987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:57.887185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:57.888908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:57.905765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c5292-732c-4076-b2de-765287c998fe None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:57.906671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 331/658] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:57.908109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:57.947048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:57.959905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:57.961477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:57.961940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:57.962381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:57.974026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:57.977288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:58.000604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:58.002464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:58.003743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:58.003743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:58.003743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:58.003743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:58.006011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 328/659] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:58.007298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:58.041868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114bebb-4d0e-44fe-ba43-ba0165c02097 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.049817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:58.088084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.090125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.090547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.090905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.104010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:58.105810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:58.128295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:58.129809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:58.131172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:58.131172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:58.131172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:58.131172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:58.134287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 332/660] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:58.135317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:58.148800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd194d1-7bb3-4aff-8f68-be6ba2864c63 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.195209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.196491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.196946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.197388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.207115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:58.208860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:50:58.238307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:58.239469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 30 11:50:58.240943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:58.244474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 329/661] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:58.245904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:58.280423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090c89f-6ec9-4011-891f-8101bac76455 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.309103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:58.329260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.329809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.330128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.330348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.331833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:58.331833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:58.333254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.349395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 333/662] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:58.355074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:58.376086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:58.402195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e563e5-e0ca-4a73-8c8f-23044f20e050 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.456683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282c978-7448-465a-9bad-493eda37d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.457527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c978-7448-465a-9bad-493eda37d909 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.457971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c978-7448-465a-9bad-493eda37d909 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.458536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282c978-7448-465a-9bad-493eda37d909 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.459470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282c978-7448-465a-9bad-493eda37d909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:58.460961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282c978-7448-465a-9bad-493eda37d909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:58.462681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282c978-7448-465a-9bad-493eda37d909 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.484217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 330/663] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:58.484217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe6741-c39e-475c-abbc-c3c75dd9ce47 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.484217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe6741-c39e-475c-abbc-c3c75dd9ce47 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.484217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe6741-c39e-475c-abbc-c3c75dd9ce47 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.484217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 331/664] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:58.484217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe6741-c39e-475c-abbc-c3c75dd9ce47 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:58.512918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdcea7-b6f0-4169-b4db-b5b9749a4867 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.555640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.556369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.556656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.556934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.557874 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:58.558190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:58.559740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.586616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 334/665] 10.222.0.32 () {56 vars in 1139 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:58.587935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:58.607168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:58.612496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe6741-c39e-475c-abbc-c3c75dd9ce47 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.651401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d994c2-211c-4415-9f89-359b5d025889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.652248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d994c2-211c-4415-9f89-359b5d025889 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.652533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d994c2-211c-4415-9f89-359b5d025889 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.652808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d994c2-211c-4415-9f89-359b5d025889 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.660130 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d994c2-211c-4415-9f89-359b5d025889 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:58.661599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d994c2-211c-4415-9f89-359b5d025889 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-20d994c2-211c-4415-9f89-359b5d025889 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:58.675967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:58.676937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:58.677847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:58.677847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:58.677847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:58.677847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:58.677847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:58.677847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:58.677847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:58.677847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:58.681979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 332/666] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:58.683268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f0f8e-baaa-4f22-85a2-0d6101ea5760 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.683557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f0f8e-baaa-4f22-85a2-0d6101ea5760 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.683833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f0f8e-baaa-4f22-85a2-0d6101ea5760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.701348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-059f0f8e-baaa-4f22-85a2-0d6101ea5760 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:58.702524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-059f0f8e-baaa-4f22-85a2-0d6101ea5760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:58.707955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:58.734985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3714dd-d65e-48b6-ae02-7c38712e0285 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.774657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.775466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.775747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.776021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.776599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 333/667] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:58.777621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f0f8e-baaa-4f22-85a2-0d6101ea5760 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:58.793902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:58.796359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:50:58.814934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:58.815986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:58.817111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:50:58.817111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:50:58.817111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:50:58.817111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:50:58.817111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:50:58.817111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:50:58.817111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:50:58.817111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:58.819257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 335/668] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:58.820298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:58.890147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:58.905350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f0f8e-baaa-4f22-85a2-0d6101ea5760 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.933447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff886a43-7d5c-4f13-9ebf-e05cfb68ebb1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:58.942996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65868c2a-62bb-4f83-bcc7-38b7c26e1f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.943744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868c2a-62bb-4f83-bcc7-38b7c26e1f5f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.944034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868c2a-62bb-4f83-bcc7-38b7c26e1f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.944345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65868c2a-62bb-4f83-bcc7-38b7c26e1f5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.971617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:58.972480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:58.972792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:58.973063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:58.974015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:58.974347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:58.976426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:58.979796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:58.996880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 336/669] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:58.997785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.026454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65868c2a-62bb-4f83-bcc7-38b7c26e1f5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:59.028313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65868c2a-62bb-4f83-bcc7-38b7c26e1f5f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:59.106786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a00c9-d461-4e32-b5e4-5816f8c0cdb8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:59.123744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:59.141349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 334/670] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:50:59.142813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65868c2a-62bb-4f83-bcc7-38b7c26e1f5f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.152961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642e327-8312-4825-a230-5d58b1cd9870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:59.153990 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642e327-8312-4825-a230-5d58b1cd9870 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.154198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642e327-8312-4825-a230-5d58b1cd9870 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.154463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642e327-8312-4825-a230-5d58b1cd9870 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.167351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642e327-8312-4825-a230-5d58b1cd9870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3642e327-8312-4825-a230-5d58b1cd9870 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:50:59.175757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:59.176754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:50:59.177570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:50:59.183336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 337/671] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:59.184226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642e327-8312-4825-a230-5d58b1cd9870 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.243877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:59.254627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65868c2a-62bb-4f83-bcc7-38b7c26e1f5f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:59.292418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:59.292739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642e327-8312-4825-a230-5d58b1cd9870 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:59.293153 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.293388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.293637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.294433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:59.294748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:59.296218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:59.313240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 335/672] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:59.313240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.337137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:59.337847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.338132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.338409 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.339239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=ad37bdce39f64836b0cd14613cabe91f) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:59.339555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:59.341028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:59.352437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 338/673] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/role_assignments?user.id=ad37bdce39f64836b0cd14613cabe91f&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:59.353926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.367601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:59.447550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80e42a-29fd-4154-be09-b0ead65b2acb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:59.477052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1842fef-9524-442a-ab3d-c0ec275eac8d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:59.492023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:59.492998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.493595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.493595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.500142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:59.507533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:59.510354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:59.518618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e49bfe-21d5-4c21-88b3-1884cb4852e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:59.519304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e49bfe-21d5-4c21-88b3-1884cb4852e5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.519584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e49bfe-21d5-4c21-88b3-1884cb4852e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.519908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e49bfe-21d5-4c21-88b3-1884cb4852e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.520665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e49bfe-21d5-4c21-88b3-1884cb4852e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:59.520972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e49bfe-21d5-4c21-88b3-1884cb4852e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:59.522362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e49bfe-21d5-4c21-88b3-1884cb4852e5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:59.530718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:59.531725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:59.532667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:59.532667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:59.532667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:59.532667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:59.532667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 336/674] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:59.533437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.542537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 339/675] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:59.542537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dbd15-58f4-4aec-8628-042a312e8fcc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.542537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dbd15-58f4-4aec-8628-042a312e8fcc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.542537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dbd15-58f4-4aec-8628-042a312e8fcc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.542537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 340/676] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:50:59.542537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dbd15-58f4-4aec-8628-042a312e8fcc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.667308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:59.670756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8701528-4d43-4d6e-bc28-93356348cdf2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:59.673945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dbd15-58f4-4aec-8628-042a312e8fcc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:59.707238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:59.707828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.708114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.708311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a2ff9-9b17-4fb7-8cf1-5a6e026e270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:59.708412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.708930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a2ff9-9b17-4fb7-8cf1-5a6e026e270c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.709217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a2ff9-9b17-4fb7-8cf1-5a6e026e270c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.709314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=7c3c27e908e34977b248042f96c70dd1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:59.709508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a2ff9-9b17-4fb7-8cf1-5a6e026e270c None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.709602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:59.711234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a2ff9-9b17-4fb7-8cf1-5a6e026e270c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:50:59.711649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:59.711742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a2ff9-9b17-4fb7-8cf1-5a6e026e270c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:59.713098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a2ff9-9b17-4fb7-8cf1-5a6e026e270c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:59.724569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 337/677] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/role_assignments?user.id=7c3c27e908e34977b248042f96c70dd1&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:59.726565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.731639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 341/678] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:50:59.733029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186f476-8f27-409a-9ea5-8e3e981910d9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.733329 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186f476-8f27-409a-9ea5-8e3e981910d9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.733621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186f476-8f27-409a-9ea5-8e3e981910d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.753834 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c186f476-8f27-409a-9ea5-8e3e981910d9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:50:59.755070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c186f476-8f27-409a-9ea5-8e3e981910d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:50:59.813492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:50:59.853997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 342/679] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:50:59.854920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186f476-8f27-409a-9ea5-8e3e981910d9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.869394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3d2bb-509f-49d7-8353-33b40f42b903 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:50:59.927770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:50:59.930217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:50:59.931409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:50:59.932598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:50:59.952023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:50:59.953895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:50:59.984980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:50:59.986194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:50:59.987290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:50:59.987290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:50:59.987290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:50:59.987290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:50:59.990304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 338/680] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:50:59.991857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:50:59.992440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186f476-8f27-409a-9ea5-8e3e981910d9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.024309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:00.038021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.038894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.039184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.039463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.047669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:00.049494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:00.064082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:00.065303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:00.066423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:00.066423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:00.066423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:00.066423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:00.066423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:00.066423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:00.066423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:00.066423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:00.067735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 343/681] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:00.072190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.114810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:00.134094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f30a7-7c60-4ebc-8721-223c964ec515 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.173197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8d2cf-c6a0-4fac-a50f-d910abb56b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.174137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8d2cf-c6a0-4fac-a50f-d910abb56b32 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.174427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8d2cf-c6a0-4fac-a50f-d910abb56b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.174706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8d2cf-c6a0-4fac-a50f-d910abb56b32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.186512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8d2cf-c6a0-4fac-a50f-d910abb56b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0af8d2cf-c6a0-4fac-a50f-d910abb56b32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:00.195527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:00.196839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:00.197810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 339/682] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:50:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:00.198504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8d2cf-c6a0-4fac-a50f-d910abb56b32 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.206511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb765d79-df64-4043-a913-70c4ab768abf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.256540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.257526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.258071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.258505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.259384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:00.259843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:00.261465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:00.277188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 344/683] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:00.278798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.299280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:00.327122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8d2cf-c6a0-4fac-a50f-d910abb56b32 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.364418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.365588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.365588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.365787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.367063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:00.367378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:00.368898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:00.388203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 340/684] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:00.388686 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.390534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:00.400954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976f518-ffc4-4426-b1ab-ff10f3b8f677 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.438198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec32773-e816-403d-af96-52a641d5386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.439025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32773-e816-403d-af96-52a641d5386b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.439314 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32773-e816-403d-af96-52a641d5386b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.439593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32773-e816-403d-af96-52a641d5386b None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.440381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec32773-e816-403d-af96-52a641d5386b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:00.440696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec32773-e816-403d-af96-52a641d5386b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:00.442332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec32773-e816-403d-af96-52a641d5386b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:00.451846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 345/685] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:00.453142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec32773-e816-403d-af96-52a641d5386b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.496773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f22a-30b1-4164-ab8a-09b32092efda None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.533756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.534903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.535398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.535906 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.539904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:00.545965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:00.547819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:00.565700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:00.566723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 30 11:51:00.567652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:00.578186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 341/686] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:00.579449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.583387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec32773-e816-403d-af96-52a641d5386b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.630017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.630683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.630968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.631246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.632620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=15ea38389eee4472a8917023643be9b1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:00.632939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:00.635093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:00.643844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 346/687] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/role_assignments?user.id=15ea38389eee4472a8917023643be9b1&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:00.644846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.668551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:00.772490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3951b-a376-4566-80ee-6a0c32bc8dc0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.813357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.813959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.814159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083cab3-8fdc-4c45-af8c-ce58bed5acff None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:00.814267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.814510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.815364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:00.815981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:00.817491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:00.839898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 347/688] 10.222.0.32 () {56 vars in 1135 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:00.845676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.885545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:00.886367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:00.886648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:00.886921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:00.906012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:00.907685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:00.928734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:00.930104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:00.931241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:00.931241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:00.931241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:00.931241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:00.936181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 342/689] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:00.937786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:00.978806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:01.003743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:01.012801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ae53c-fa20-4528-bff5-c58d37ac7038 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:01.055297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96460-c86f-431c-a717-6623f4f190fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:01.055976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96460-c86f-431c-a717-6623f4f190fb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.056365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96460-c86f-431c-a717-6623f4f190fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.056564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96460-c86f-431c-a717-6623f4f190fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.065388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeca2ba-1e27-4f1f-a754-369978d9ee59 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:01.103976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:01.105132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.105619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.106122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.107150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:01.107679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:01.109387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:01.123264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 343/690] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:01.133416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:01.143651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a96460-c86f-431c-a717-6623f4f190fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:01.145236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a96460-c86f-431c-a717-6623f4f190fb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:01.237360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:01.249161 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a20be0-ab77-4382-99cb-a9fa69f8d1be None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:01.268554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:01.284408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 348/691] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:51:01.288599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:01.290259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96460-c86f-431c-a717-6623f4f190fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:01.292924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.294319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.294828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.312020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:01.314495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:01.346442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:01.347515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:01.348539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:01.353348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 344/692] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:51:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:01.361496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:01.468018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96460-c86f-431c-a717-6623f4f190fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:01.512396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d3804-7042-400b-a10f-f83a2670e7b8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:01.535675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:01.536522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.536813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.537120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.551083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:01.552346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.552877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.553363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.554495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:01.555033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:01.556812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:01.577620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:01.578761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:01.582461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 345/693] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:51:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:01.589581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:01.613388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:01.614319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:01.615224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:01.615224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:01.615224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:01.615224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:01.620326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 349/694] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:01.629877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:01.642405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:01.696217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:01.700212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c645c-80c4-4b54-b9d2-48ab931018d2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:01.738343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46e3e2-328f-4792-94b3-6f6df302f3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:01.739102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e3e2-328f-4792-94b3-6f6df302f3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.739459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e3e2-328f-4792-94b3-6f6df302f3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.739812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46e3e2-328f-4792-94b3-6f6df302f3e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.814319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46e3e2-328f-4792-94b3-6f6df302f3e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:01.815916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46e3e2-328f-4792-94b3-6f6df302f3e1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:01.852644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:01.862609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f77082-d51d-4db9-9ec7-48b2707743d6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:01.929719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 346/695] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 30 11:51:01.931154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd540e-d97f-4388-a0eb-a3586953f27e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.931518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd540e-d97f-4388-a0eb-a3586953f27e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.931869 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd540e-d97f-4388-a0eb-a3586953f27e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.933495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 347/696] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:01.934922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62a5be-cc93-4ef1-9c1b-a0b7955693cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:01.935764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a5be-cc93-4ef1-9c1b-a0b7955693cb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.936687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd540e-d97f-4388-a0eb-a3586953f27e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:01.937270 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a5be-cc93-4ef1-9c1b-a0b7955693cb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.937618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a5be-cc93-4ef1-9c1b-a0b7955693cb None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.938281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62a5be-cc93-4ef1-9c1b-a0b7955693cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:01.938593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62a5be-cc93-4ef1-9c1b-a0b7955693cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:01.939997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62a5be-cc93-4ef1-9c1b-a0b7955693cb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:01.957133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 350/697] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:01.958946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba1292-22f9-47d3-bb6c-5c2f3beddb5c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:01.959246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba1292-22f9-47d3-bb6c-5c2f3beddb5c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:01.959532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba1292-22f9-47d3-bb6c-5c2f3beddb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:01.981015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0bba1292-22f9-47d3-bb6c-5c2f3beddb5c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:01.982240 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0bba1292-22f9-47d3-bb6c-5c2f3beddb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:02.065404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd540e-d97f-4388-a0eb-a3586953f27e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.077149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 351/698] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:51:02.077149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba1292-22f9-47d3-bb6c-5c2f3beddb5c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.102946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:02.111952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:02.122092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.123273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.123797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.124327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.135803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.137487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:02.155744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:02.156848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:02.157736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:02.157736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:02.157736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:02.157736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:02.159424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 348/699] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:02.160590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.211839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba1292-22f9-47d3-bb6c-5c2f3beddb5c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.255643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.256813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.257346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.258105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.266313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.268295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:02.284684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:02.285627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:02.286502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:02.286502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:02.286502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:02.286502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:02.286502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:02.286502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:02.286502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:02.286502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:02.286925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870e419-130e-412d-aa61-d83b40d6f849 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.290341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 352/700] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:02.291517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.325472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.326317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.326609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.326899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.338716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.340334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:02.359693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:02.360717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:02.361696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:02.361696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:02.361696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:02.361696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:02.361696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 349/701] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:02.361982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.390925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:02.395546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:02.416490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd24ca-5a7b-4d29-8c1a-0be2d4e249d2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.457903 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadb8e3-56a3-46a8-b34b-6e2e981708ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.458570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadb8e3-56a3-46a8-b34b-6e2e981708ef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.458855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadb8e3-56a3-46a8-b34b-6e2e981708ef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.459130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadb8e3-56a3-46a8-b34b-6e2e981708ef None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.459851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadb8e3-56a3-46a8-b34b-6e2e981708ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:02.460775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadb8e3-56a3-46a8-b34b-6e2e981708ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.462716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadb8e3-56a3-46a8-b34b-6e2e981708ef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.479084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 353/702] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:02.479084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b913fd-fcda-4b16-ac62-c4578f8a647b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.479084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b913fd-fcda-4b16-ac62-c4578f8a647b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.479084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b913fd-fcda-4b16-ac62-c4578f8a647b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.479084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 354/703] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:02.479950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b913fd-fcda-4b16-ac62-c4578f8a647b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.484880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd600231-25cf-4365-aa8c-8fad57a3a26b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.531857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.533304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.533592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.533877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.534669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:02.534985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.536452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.554848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 350/704] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:02.555803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.602382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b913fd-fcda-4b16-ac62-c4578f8a647b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.638764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d9274-5bd1-4637-83c2-df13eb24cd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.639436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9274-5bd1-4637-83c2-df13eb24cd05 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.639715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9274-5bd1-4637-83c2-df13eb24cd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.639988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9274-5bd1-4637-83c2-df13eb24cd05 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.640738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d9274-5bd1-4637-83c2-df13eb24cd05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:02.641135 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d9274-5bd1-4637-83c2-df13eb24cd05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.642670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d9274-5bd1-4637-83c2-df13eb24cd05 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.650077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:02.656960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 355/705] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:02.658421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a655b6f-b3ab-400a-b8c5-5df241e11997 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.658877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a655b6f-b3ab-400a-b8c5-5df241e11997 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.659312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a655b6f-b3ab-400a-b8c5-5df241e11997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.665935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:02.676118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2a655b6f-b3ab-400a-b8c5-5df241e11997 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:02.676720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686eac3-64d1-4128-b717-51fcfdc72a23 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.677333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2a655b6f-b3ab-400a-b8c5-5df241e11997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:02.714086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.714813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.715080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.715355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.722612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.724067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:02.738975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:02.739899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:02.740829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 351/706] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:02.741894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.763984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 356/707] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:51:02.763984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a655b6f-b3ab-400a-b8c5-5df241e11997 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.853299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b892879-70f3-4e55-9f4e-81665d5720a8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.874833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a655b6f-b3ab-400a-b8c5-5df241e11997 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:02.890059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.890729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.891013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.891295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.892405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=e0cb45c1eb6040548634022673a51e56) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:02.892719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.894144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.901085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 352/708] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/role_assignments?user.id=e0cb45c1eb6040548634022673a51e56&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:02.901900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:02.919172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:02.919759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:02.932731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cae68-5c06-4bd5-b012-9f2975bf5312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:02.934681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cae68-5c06-4bd5-b012-9f2975bf5312 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:02.935224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cae68-5c06-4bd5-b012-9f2975bf5312 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:02.935973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7cae68-5c06-4bd5-b012-9f2975bf5312 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:02.937522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cae68-5c06-4bd5-b012-9f2975bf5312 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:02.939854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7cae68-5c06-4bd5-b012-9f2975bf5312 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:02.976365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 357/709] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:51:02 2022] POST /identity/v3/users => generated 254 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:02.976365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cae68-5c06-4bd5-b012-9f2975bf5312 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:03.093370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912cefd-4abd-4ac1-a6ef-ba6b13967fda None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:03.164965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:03.165834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:03.166120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:03.166401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:03.167261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:03.167575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:03.169373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:03.176307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7cae68-5c06-4bd5-b012-9f2975bf5312 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:03.184417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 353/710] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:03.191076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:03.229190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:03.229614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:03.229998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:03.230373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:03.231167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:03.232802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:03.249866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:03.251077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:03.251931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:03.253379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 358/711] 10.222.0.32 () {56 vars in 1129 bytes} [Mon May 30 11:51:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:03.260777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:03.303239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:03.325192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:03.339747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621538bc-24d5-4f08-ae5a-0683064cc0ff None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:03.385438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:03.386952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:03.387527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:03.387994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:03.389509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=70dce2f0284847ab8f07f55d700d0f60) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:03.389837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:03.391579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:03.398355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ef403-a949-44c3-8afc-c39a4441f3ac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:03.400055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 354/712] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:03 2022] GET /identity/v3/role_assignments?user.id=70dce2f0284847ab8f07f55d700d0f60&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=4d21498572b14478a996ac538be5382e => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:03.443859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab714b2-70ff-4fb3-aa90-020de4d5e198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:03.444700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab714b2-70ff-4fb3-aa90-020de4d5e198 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:03.445011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab714b2-70ff-4fb3-aa90-020de4d5e198 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:03.445322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab714b2-70ff-4fb3-aa90-020de4d5e198 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:03.446182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab714b2-70ff-4fb3-aa90-020de4d5e198 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:03.446507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab714b2-70ff-4fb3-aa90-020de4d5e198 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:03.447938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab714b2-70ff-4fb3-aa90-020de4d5e198 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:03.453543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 359/713] 10.222.0.32 () {56 vars in 1124 bytes} [Mon May 30 11:51:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:03.459093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:03.594009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d9442-6f8d-433a-be9a-7938230b95fc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:03.594905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d9442-6f8d-433a-be9a-7938230b95fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:03.595393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d9442-6f8d-433a-be9a-7938230b95fc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:03.597050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 360/714] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:03.603733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4752-8f65-4bdf-87d7-44025e816220 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:03.604112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4752-8f65-4bdf-87d7-44025e816220 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:03.604501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4752-8f65-4bdf-87d7-44025e816220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:03.618455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3645f97-dc9d-4833-ad67-1cf376509271 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:03.641313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d14d4752-8f65-4bdf-87d7-44025e816220 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:03.642821 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d14d4752-8f65-4bdf-87d7-44025e816220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:03.658783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:03.659855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:03.660134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:03.660433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:03.661148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:03.661456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:03.662896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:03.676030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 355/715] 10.222.0.32 () {56 vars in 1124 bytes} [Mon May 30 11:51:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:03.745511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 361/716] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:51:03.768216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:03.771044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:03.913688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b5abf-fced-4cb9-8cbe-f5fa1ace2b6c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:03.960880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:03.961930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:03.962378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:03.962792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] PATH_INFO: `/v3/services/network` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:03.963608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:03.965158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:03.985489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:03.986791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:03.987820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:03.990611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 356/717] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:03 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:03.996830 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d4752-8f65-4bdf-87d7-44025e816220 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:04.068609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:04.127728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d4752-8f65-4bdf-87d7-44025e816220 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:04.164828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213085f-c51a-440a-8428-1f69a7d7a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:04.165427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213085f-c51a-440a-8428-1f69a7d7a339 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:04.165708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213085f-c51a-440a-8428-1f69a7d7a339 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:04.165988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213085f-c51a-440a-8428-1f69a7d7a339 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:04.166703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213085f-c51a-440a-8428-1f69a7d7a339 None admin] RBAC: Adding query filter params (name=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:04.167012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213085f-c51a-440a-8428-1f69a7d7a339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:04.168419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213085f-c51a-440a-8428-1f69a7d7a339 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:04.173681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 362/718] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:04.181259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:04.260581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:04.298635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d8a46-cef2-4528-938e-f076a156e969 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:04.334551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d6a90-f705-4caa-9594-a6a5b2f2ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:04.335812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d6a90-f705-4caa-9594-a6a5b2f2ab5b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:04.336095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d6a90-f705-4caa-9594-a6a5b2f2ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:04.336400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d6a90-f705-4caa-9594-a6a5b2f2ab5b None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:04.337120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d6a90-f705-4caa-9594-a6a5b2f2ab5b None admin] RBAC: Adding query filter params (type=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:04.337429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d6a90-f705-4caa-9594-a6a5b2f2ab5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:04.338810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d6a90-f705-4caa-9594-a6a5b2f2ab5b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:04.344071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 357/719] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:05.975210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74cc60-8508-474a-ae3b-08194f7babc5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:05.976931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74cc60-8508-474a-ae3b-08194f7babc5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:05.977417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74cc60-8508-474a-ae3b-08194f7babc5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:05.979244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 363/720] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:05.984433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418bb9e-3a77-4e9d-91c2-5406eb117abd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:05.985310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418bb9e-3a77-4e9d-91c2-5406eb117abd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:05.985806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418bb9e-3a77-4e9d-91c2-5406eb117abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.008713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8418bb9e-3a77-4e9d-91c2-5406eb117abd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:06.010264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8418bb9e-3a77-4e9d-91c2-5406eb117abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:06.087326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 358/721] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:51:06.124394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74cc60-8508-474a-ae3b-08194f7babc5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:06.227153 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d2630-043b-4ae1-a6a5-3785583e6b44 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.227539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d2630-043b-4ae1-a6a5-3785583e6b44 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.227705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d2630-043b-4ae1-a6a5-3785583e6b44 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.233025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 359/722] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:06.234548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db579d2-77cc-4efc-ad4c-77aae2bacc0c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.234839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db579d2-77cc-4efc-ad4c-77aae2bacc0c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.235121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db579d2-77cc-4efc-ad4c-77aae2bacc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.245002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74cc60-8508-474a-ae3b-08194f7babc5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:06.260528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:06.265859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0db579d2-77cc-4efc-ad4c-77aae2bacc0c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:06.267326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0db579d2-77cc-4efc-ad4c-77aae2bacc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:06.301118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64be3c-f7e7-49c1-a542-f24d067183f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:06.302455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64be3c-f7e7-49c1-a542-f24d067183f1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.302954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64be3c-f7e7-49c1-a542-f24d067183f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.303424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64be3c-f7e7-49c1-a542-f24d067183f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.318675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64be3c-f7e7-49c1-a542-f24d067183f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:06.329913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ad64be3c-f7e7-49c1-a542-f24d067183f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:06.331949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:06.333263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:06.334102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:06.336980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 364/723] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:06.338889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77436341-320a-411e-aa6a-fbb871974f92 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.339180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77436341-320a-411e-aa6a-fbb871974f92 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.339461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77436341-320a-411e-aa6a-fbb871974f92 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.341192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 365/724] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:06.342254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77436341-320a-411e-aa6a-fbb871974f92 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:06.354941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 360/725] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 30 11:51:06.359241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babca42-5e36-42e0-be0a-b24623b6c0d8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.359241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babca42-5e36-42e0-be0a-b24623b6c0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.359241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babca42-5e36-42e0-be0a-b24623b6c0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.389595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9babca42-5e36-42e0-be0a-b24623b6c0d8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:06.390988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9babca42-5e36-42e0-be0a-b24623b6c0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:06.471484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77436341-320a-411e-aa6a-fbb871974f92 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:06.480520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 361/726] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:51:06.481532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babca42-5e36-42e0-be0a-b24623b6c0d8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:06.510576 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:06.511258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.511538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.512123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.512753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:06.513060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:06.514581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:06.530893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 366/727] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:06.530893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:06.562665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:06.616450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babca42-5e36-42e0-be0a-b24623b6c0d8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:06.640075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:06.651604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc7c05-a9fb-471d-a41d-6103443fed65 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:06.656641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228001ff-7f85-4559-9cd3-41bac4ef8e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:06.657761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228001ff-7f85-4559-9cd3-41bac4ef8e9c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.658246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228001ff-7f85-4559-9cd3-41bac4ef8e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.658717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228001ff-7f85-4559-9cd3-41bac4ef8e9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.671855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228001ff-7f85-4559-9cd3-41bac4ef8e9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-228001ff-7f85-4559-9cd3-41bac4ef8e9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:06.679998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:06.681002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:06.681844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 362/728] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:51:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:06.682551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228001ff-7f85-4559-9cd3-41bac4ef8e9c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:06.695652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3987a60-e362-4e98-8035-323b0410ebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:06.696773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3987a60-e362-4e98-8035-323b0410ebf0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.696956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3987a60-e362-4e98-8035-323b0410ebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.697224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3987a60-e362-4e98-8035-323b0410ebf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.708253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3987a60-e362-4e98-8035-323b0410ebf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e3987a60-e362-4e98-8035-323b0410ebf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:06.716494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:06.717440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:06.718354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 367/729] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:06.719075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5e7fa-a35c-4321-8f5d-7a4567c3f730 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.719764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5e7fa-a35c-4321-8f5d-7a4567c3f730 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.719764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5e7fa-a35c-4321-8f5d-7a4567c3f730 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.725076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 368/730] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:06.725076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5e7fa-a35c-4321-8f5d-7a4567c3f730 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:06.799609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228001ff-7f85-4559-9cd3-41bac4ef8e9c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:06.823129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:06.838231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:06.838499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5e7fa-a35c-4321-8f5d-7a4567c3f730 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:06.839070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.839352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.839632 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.854283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:06.855809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:06.874773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:06.875953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:06.877030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:06.877030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:06.877030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:06.877030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:06.877030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 363/731] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:06.877337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:06.886330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111aa6a-903e-4591-8e81-3566cad873c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:06.887027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111aa6a-903e-4591-8e81-3566cad873c3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.887320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111aa6a-903e-4591-8e81-3566cad873c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.887607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111aa6a-903e-4591-8e81-3566cad873c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.888474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111aa6a-903e-4591-8e81-3566cad873c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:06.888793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111aa6a-903e-4591-8e81-3566cad873c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:06.890193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111aa6a-903e-4591-8e81-3566cad873c3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:06.900470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 369/732] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:06.901799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f1492-ea27-4d0a-920f-e6d88cb90517 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:06.902096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f1492-ea27-4d0a-920f-e6d88cb90517 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:06.902382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f1492-ea27-4d0a-920f-e6d88cb90517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:06.919642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:06.919848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bf6f1492-ea27-4d0a-920f-e6d88cb90517 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:06.921340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bf6f1492-ea27-4d0a-920f-e6d88cb90517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:06.996359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 370/733] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:51:06.996359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f1492-ea27-4d0a-920f-e6d88cb90517 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:06.997075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c5053-e289-4ca3-b408-6041f2b4a500 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.034140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.034816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.035110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.035397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.036173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:07.036493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.037912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.046782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 364/734] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:07.047719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.097914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:07.122239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f1492-ea27-4d0a-920f-e6d88cb90517 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.151732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:07.159165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfde32-6ba0-4d69-824d-b260c2725833 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.160973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.162084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.162084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.162268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.163010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:07.163277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.164791 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.182527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 371/735] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:07.182527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.195823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.196682 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.196969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.197243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.207800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.209279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:07.226356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:07.227285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:07.228235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:07.228235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:07.228235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:07.228235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:07.228235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 365/736] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:07.228511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.291745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913f0ab-338f-4e3f-9c66-999afd2af20b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.327049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb0372-6558-4314-9df9-e2a4a8f01ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.327713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb0372-6558-4314-9df9-e2a4a8f01ad3 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.327814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8aac7-ce25-4c26-a5fc-b647cac0cd51 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.328001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb0372-6558-4314-9df9-e2a4a8f01ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.328315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb0372-6558-4314-9df9-e2a4a8f01ad3 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.329042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb0372-6558-4314-9df9-e2a4a8f01ad3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.330422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb0372-6558-4314-9df9-e2a4a8f01ad3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.340572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:07.348595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 372/737] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:51:07 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:07.350369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db66d0-9074-42b9-a3bc-77a2473503a7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.351155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db66d0-9074-42b9-a3bc-77a2473503a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.351500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db66d0-9074-42b9-a3bc-77a2473503a7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.353155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 373/738] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:07.353878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db66d0-9074-42b9-a3bc-77a2473503a7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.386251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.387375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.387733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.388115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.400775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.402265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.404394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:07.429259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:07.430438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:07.432303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:07.432303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:07.432303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:07.432303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:07.432303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 366/739] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:07.432303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.490808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db66d0-9074-42b9-a3bc-77a2473503a7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.532543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb9733-452c-42b7-bd05-002a56282f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.533483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9733-452c-42b7-bd05-002a56282f82 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.533876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9733-452c-42b7-bd05-002a56282f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.534064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb9733-452c-42b7-bd05-002a56282f82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.544031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb9733-452c-42b7-bd05-002a56282f82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.545736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb9733-452c-42b7-bd05-002a56282f82 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.556861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f2443-03e5-4fb9-aaac-759056bbed14 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6eeb9733-452c-42b7-bd05-002a56282f82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:07.562498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:07.563458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:07.564500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 374/740] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:07.565093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0975-f510-4135-8dea-f41b9557720a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.566076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0975-f510-4135-8dea-f41b9557720a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.566076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0975-f510-4135-8dea-f41b9557720a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.584207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8b6a0975-f510-4135-8dea-f41b9557720a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:07.585824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8b6a0975-f510-4135-8dea-f41b9557720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:07.602152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.602824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.603117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.603397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.604187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:07.604512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.605911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.624948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 367/741] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:07.624948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.660194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:07.673430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 375/742] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 30 11:51:07.674429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a0975-f510-4135-8dea-f41b9557720a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.691231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:07.740383 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62ce33-8a2e-4d2f-b6c7-e61a637bc2b0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.777744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.778429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.778717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.779006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.779740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:07.780052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.781497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.785623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a0975-f510-4135-8dea-f41b9557720a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.800135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 368/743] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:07.801019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.826763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.827382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.827674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.827970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.828839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:07.829155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.830619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.857781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 376/744] 10.222.0.32 () {56 vars in 1135 bytes} [Mon May 30 11:51:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:07.857781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:07.916087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:07.924658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0ebe8-ad10-4442-9089-a0adce61f674 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.940965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:07.966684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:07.967801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:07.968090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:07.968393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:07.978114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:07.979583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:07.979757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47430b4-e428-466e-bf03-9e44af993589 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:07.996427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:07.997901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 369/745] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:51:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:07.998840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:08.023487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c471447-9e8c-40be-b2b1-b76b28dd09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.024849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471447-9e8c-40be-b2b1-b76b28dd09fd None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.025318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471447-9e8c-40be-b2b1-b76b28dd09fd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.025603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471447-9e8c-40be-b2b1-b76b28dd09fd None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.026329 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c471447-9e8c-40be-b2b1-b76b28dd09fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:08.027718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c471447-9e8c-40be-b2b1-b76b28dd09fd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:08.043952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 377/746] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:51:07 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:08.044858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c471447-9e8c-40be-b2b1-b76b28dd09fd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:08.141614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc673a66-2763-4e90-a5c6-9da6d7b51166 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:08.189924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.190552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c471447-9e8c-40be-b2b1-b76b28dd09fd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:08.190779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.191065 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.191345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.196738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:08.204145 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:08.205985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:08.229672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:08.230692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:08.231650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:08.236030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 370/747] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:51:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:08.237230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:08.245806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489f75-9ce9-4e85-aaf2-65e29d3c5527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.246871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489f75-9ce9-4e85-aaf2-65e29d3c5527 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.247363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489f75-9ce9-4e85-aaf2-65e29d3c5527 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.247869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489f75-9ce9-4e85-aaf2-65e29d3c5527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.279540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:08.342588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca489f75-9ce9-4e85-aaf2-65e29d3c5527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:08.344107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca489f75-9ce9-4e85-aaf2-65e29d3c5527 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:08.377461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf1bf-df14-434e-bd5d-2557f4b6d946 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:08.425130 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.426163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.426985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.427474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.429615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:08.430149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:08.432577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:08.454802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 371/748] 10.222.0.32 () {56 vars in 1149 bytes} [Mon May 30 11:51:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:08.459449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:08.459614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 378/749] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:08.460510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489f75-9ce9-4e85-aaf2-65e29d3c5527 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:08.470099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:08.557083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:08.569957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245550e-dace-4aec-af9b-caef9f8d9b42 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:08.578957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489f75-9ce9-4e85-aaf2-65e29d3c5527 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:08.611045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.611784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.612079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.612387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.613240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:08.613560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:08.615261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:08.626887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0a28a-519e-496b-a828-513640b99038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.627881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0a28a-519e-496b-a828-513640b99038 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.628437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0a28a-519e-496b-a828-513640b99038 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.628917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0a28a-519e-496b-a828-513640b99038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.641410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 372/750] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:51:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:08.641410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:08.720123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0a28a-519e-496b-a828-513640b99038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:08.721973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0a28a-519e-496b-a828-513640b99038 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:08.726141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:08.762255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009861-70c5-463e-a5e2-e7bca5a3d5d3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:08.799935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.801176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.801176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.801467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.813009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:08.814866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:08.822237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:08.836183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:08.837123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:08.838060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:08.838060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:08.838060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:08.838060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:08.838060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 373/751] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:08.838787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:08.840795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 379/752] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:08.842850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0a28a-519e-496b-a828-513640b99038 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:08.953941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2fec3-3688-433d-afa6-65cea56c84b4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:08.955270 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0a28a-519e-496b-a828-513640b99038 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:08.958575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:08.994291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.994768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:08.995018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.995306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.995588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.996378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:08.996880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:08.997478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:08.998504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:08.999063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.000709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.007225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.008754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.015161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 380/753] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:09.016393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:09.028551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:09.029514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:09.030410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:09.030410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:09.030410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:09.030410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:09.034452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 374/754] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:09.035306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.090985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:09.134654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431fe1c-0be0-4d29-bbfe-96af5e7bdfa7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.149960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34af9d-4d79-4fdc-9b19-aa00ef6ebed6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.172806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.173981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.174470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.174980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.187317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.187876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.188731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.189018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.189301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.190068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.197405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.198972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:09.209966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:09.211131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:09.212121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:09.212121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:09.212121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:09.212121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:09.213349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:09.214177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:09.215051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 375/755] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:09.215941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.217187 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 381/756] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:09.220138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.241650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:09.340120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07deb8-0a3c-4d2a-b68e-84c37530b16f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.341297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6648427-a6fc-46aa-95bf-f93b2c563b8c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.354137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:09.384190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.384879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.385172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.385456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.386193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:09.386510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.387914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.389050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.390058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.390657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.391149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.392444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:09.392967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.394695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.407068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 376/757] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:09.407719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.411620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 382/758] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:09.413938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.513327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:09.521143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdb24a-9111-442d-8f36-bd586d5d0a65 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.544081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46c05b-c6ee-4f0c-8b67-8b394309fd7f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.561290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.562375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.562814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.563243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.564199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:09.564704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.566509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.585246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 377/759] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:09.586656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.602320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.603203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.603560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.603924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.611871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.613561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.621713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:09.631266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:09.632800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:09.634059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:09.634059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:09.634059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:09.634059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:09.634059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:09.634059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:09.634059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:09.634059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:09.638015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 383/760] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:09.639091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.703406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb3c12-ad75-4706-845a-d6c006c053ce None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.740648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.741900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.742181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.742726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.743303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=e0cb45c1eb6040548634022673a51e56) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:09.743680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.745081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.749047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34073c-5b51-4e9c-92fe-5ae1487274d0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.752407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 378/761] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/role_assignments?user.id=e0cb45c1eb6040548634022673a51e56&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=4d21498572b14478a996ac538be5382e => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:09.752407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.791363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396646b8-f63d-4bd5-b5d5-43201e729bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.792533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396646b8-f63d-4bd5-b5d5-43201e729bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.792988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396646b8-f63d-4bd5-b5d5-43201e729bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.793434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396646b8-f63d-4bd5-b5d5-43201e729bb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.797089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:09.804355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396646b8-f63d-4bd5-b5d5-43201e729bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.806279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396646b8-f63d-4bd5-b5d5-43201e729bb1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-396646b8-f63d-4bd5-b5d5-43201e729bb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:09.822101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:09.823889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:09.825730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:09.825730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:09.825730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:09.825730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:09.825730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:09.825730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:09.825730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:09.825730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:09.829307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 384/762] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:09.830896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad5fd-8357-4a12-bf01-e28e420c86c8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.831195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad5fd-8357-4a12-bf01-e28e420c86c8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.831469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cad5fd-8357-4a12-bf01-e28e420c86c8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.834518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 385/763] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:09.835721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cad5fd-8357-4a12-bf01-e28e420c86c8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:09.912258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:09.922578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16080690-2e3a-4d47-999f-4f690caf4923 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.970507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cad5fd-8357-4a12-bf01-e28e420c86c8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:09.973053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3dd4f-b371-49d7-8a90-389c07167b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:09.974670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3dd4f-b371-49d7-8a90-389c07167b45 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.974962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3dd4f-b371-49d7-8a90-389c07167b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.975246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3dd4f-b371-49d7-8a90-389c07167b45 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:09.977370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3dd4f-b371-49d7-8a90-389c07167b45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:09.977699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3dd4f-b371-49d7-8a90-389c07167b45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:09.979197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3dd4f-b371-49d7-8a90-389c07167b45 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:09.997103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 379/764] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:09.998759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58915b-2c8e-4d48-b565-aec987e2c15d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:09.999065 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58915b-2c8e-4d48-b565-aec987e2c15d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:09.999351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58915b-2c8e-4d48-b565-aec987e2c15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.015752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:10.016731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.017276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.017577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.018421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:10.018688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:10.020194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:10.020483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4c58915b-2c8e-4d48-b565-aec987e2c15d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:10.021672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4c58915b-2c8e-4d48-b565-aec987e2c15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:10.035002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 386/765] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:10.035906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:10.106869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:10.134911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 380/766] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:10.136201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58915b-2c8e-4d48-b565-aec987e2c15d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:10.191387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208c53b-e326-46f2-919f-5f1400e8dff9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:10.228382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3b3c-fac2-4483-bcda-bd13fb38d52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:10.229296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3b3c-fac2-4483-bcda-bd13fb38d52b None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.229583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3b3c-fac2-4483-bcda-bd13fb38d52b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.229867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3b3c-fac2-4483-bcda-bd13fb38d52b None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/70dce2f0284847ab8f07f55d700d0f60/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.231877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3b3c-fac2-4483-bcda-bd13fb38d52b None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=70dce2f0284847ab8f07f55d700d0f60, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:10.233355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3b3c-fac2-4483-bcda-bd13fb38d52b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:10.241680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:10.253025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 387/767] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:51:10 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/70dce2f0284847ab8f07f55d700d0f60/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:51:10.260463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3b3c-fac2-4483-bcda-bd13fb38d52b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:10.276098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58915b-2c8e-4d48-b565-aec987e2c15d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:10.330116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ac96c-4bc9-4f3c-b946-3f15e511088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:10.330859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ac96c-4bc9-4f3c-b946-3f15e511088a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.331203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ac96c-4bc9-4f3c-b946-3f15e511088a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.331529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ac96c-4bc9-4f3c-b946-3f15e511088a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.332592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ac96c-4bc9-4f3c-b946-3f15e511088a None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=d087549c76ae4c3896c7c8c6b5a74363) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:10.332922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ac96c-4bc9-4f3c-b946-3f15e511088a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:10.335279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ac96c-4bc9-4f3c-b946-3f15e511088a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:10.343647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 381/768] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:10 2022] GET /identity/v3/role_assignments?user.id=d087549c76ae4c3896c7c8c6b5a74363&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:10.484293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9340a10-6d5b-4c9f-993c-10c1020f8fa3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.484491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9340a10-6d5b-4c9f-993c-10c1020f8fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.484772 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9340a10-6d5b-4c9f-993c-10c1020f8fa3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.485321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3b3c-fac2-4483-bcda-bd13fb38d52b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:10.491813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 382/769] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:10.493365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613c085-8ba6-46e8-b904-3f045fa066f7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.493662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613c085-8ba6-46e8-b904-3f045fa066f7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.493948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613c085-8ba6-46e8-b904-3f045fa066f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.507252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:10.523242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c613c085-8ba6-46e8-b904-3f045fa066f7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:10.525276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c613c085-8ba6-46e8-b904-3f045fa066f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:10.530943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:10.532020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.532516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.532943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.533783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:10.535389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:10.550036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:10.551244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:10.552300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 388/770] 10.222.0.32 () {56 vars in 1129 bytes} [Mon May 30 11:51:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:10.560498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:10.621334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 383/771] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:10.645837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613c085-8ba6-46e8-b904-3f045fa066f7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:10.717758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afa29a-69a0-41db-8f2c-44c96d84243f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:10.733757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:10.755217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:10.755797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.756087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.756400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.757160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:10.757468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:10.758721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613c085-8ba6-46e8-b904-3f045fa066f7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:10.758849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:10.764392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 389/772] 10.222.0.32 () {56 vars in 1124 bytes} [Mon May 30 11:51:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:10.777433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:10.796836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:10.797935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.798374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.798802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] PATH_INFO: `/v3/services/network` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.799611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:10.801278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:10.814532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:10.815762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:10.816901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:10.819390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 384/773] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:10 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:10.825579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:10.860568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:10.913776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb9949-bb68-47bb-9183-694d448b4282 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:10.963009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a2949-c3d9-4f3e-bc46-8a4e7b7a8a69 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:10.972654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:10.973336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:10.973628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:10.973910 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:10.974743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:10.975062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:10.976844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:10.984335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 390/774] 10.222.0.32 () {56 vars in 1124 bytes} [Mon May 30 11:51:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:10.992184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:11.038675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:11.039440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:11.039630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:11.039912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:11.040663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] RBAC: Adding query filter params (name=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:11.041025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:11.042410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:11.049507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 385/775] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:11.055600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:11.064934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:11.111276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d9fe9-3e3b-47e7-9a3c-f4d968550997 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:11.121553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:11.155786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:11.156556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:11.156838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:11.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:11.157918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] RBAC: Adding query filter params (interface=public, service_id=304c8a107c6a4657836ae0c1d811315d, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:11.158233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:11.159634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:11.178507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 391/776] 10.222.0.32 () {56 vars in 1251 bytes} [Mon May 30 11:51:10 2022] GET /identity/v3/endpoints?service_id=304c8a107c6a4657836ae0c1d811315d&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:11.183973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:11.224094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a44b13-2982-4898-ad92-08c2ce5f2707 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:11.295721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:11.296423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:11.296708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:11.296989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:11.297709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] RBAC: Adding query filter params (type=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:11.298018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:11.299401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:11.309844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 386/777] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:11.316007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:11.384513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25702d9a-335e-448d-96f2-14e6b1369188 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:11.431636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:11.462948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fe784-9023-49db-b463-f94f62859431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:11.463632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fe784-9023-49db-b463-f94f62859431 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:11.463920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fe784-9023-49db-b463-f94f62859431 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:11.464227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fe784-9023-49db-b463-f94f62859431 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:11.464922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fe784-9023-49db-b463-f94f62859431 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:11.466322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fe784-9023-49db-b463-f94f62859431 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:11.480051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 392/778] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:51:11 2022] GET /identity/v3/services => generated 772 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:11.484758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ec57d-9c3d-4c53-abf6-5097bc83b309 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:11.534524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85294162-232f-4348-98f1-cf6ff1e19ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:11.535270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85294162-232f-4348-98f1-cf6ff1e19ddc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:11.535556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85294162-232f-4348-98f1-cf6ff1e19ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:11.535882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85294162-232f-4348-98f1-cf6ff1e19ddc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:11.536712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85294162-232f-4348-98f1-cf6ff1e19ddc None admin] RBAC: Adding query filter params (interface=public, service_id=1860053e511b45a3802238b2189bed4e, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:11.537034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85294162-232f-4348-98f1-cf6ff1e19ddc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:11.539035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85294162-232f-4348-98f1-cf6ff1e19ddc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:11.551341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 387/779] 10.222.0.32 () {56 vars in 1251 bytes} [Mon May 30 11:51:11 2022] GET /identity/v3/endpoints?service_id=1860053e511b45a3802238b2189bed4e&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:11.557109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fe784-9023-49db-b463-f94f62859431 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:11.597897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:11.682066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fe784-9023-49db-b463-f94f62859431 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:11.719522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f088cd2-b194-4668-8cef-f385107ce426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:11.720224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088cd2-b194-4668-8cef-f385107ce426 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:11.720517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088cd2-b194-4668-8cef-f385107ce426 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:11.720799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f088cd2-b194-4668-8cef-f385107ce426 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:11.721503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088cd2-b194-4668-8cef-f385107ce426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:11.722918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f088cd2-b194-4668-8cef-f385107ce426 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:11.730086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 393/780] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:51:11 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:12.492641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f256-6849-473c-93cb-4e190baa4b0b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:12.493359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f256-6849-473c-93cb-4e190baa4b0b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:12.493359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f256-6849-473c-93cb-4e190baa4b0b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:12.494608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 388/781] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:12.500378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a9067-1605-46ac-8ba0-431bc67c30f7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:12.501077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a9067-1605-46ac-8ba0-431bc67c30f7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:12.501077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a9067-1605-46ac-8ba0-431bc67c30f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:12.524789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-608a9067-1605-46ac-8ba0-431bc67c30f7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:12.526033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-608a9067-1605-46ac-8ba0-431bc67c30f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:12.627573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 394/782] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:12.662117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7f256-6849-473c-93cb-4e190baa4b0b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:12.805709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7f256-6849-473c-93cb-4e190baa4b0b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:12.854838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ec42b-a941-4727-938a-2e609cefea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:12.856028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec42b-a941-4727-938a-2e609cefea04 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:12.856567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec42b-a941-4727-938a-2e609cefea04 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:12.857051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec42b-a941-4727-938a-2e609cefea04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:12.859440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:12.877049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ec42b-a941-4727-938a-2e609cefea04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-657ec42b-a941-4727-938a-2e609cefea04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:12.885780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:12.886792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:12.887630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:12.894942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 389/783] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:12.899802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a9067-1605-46ac-8ba0-431bc67c30f7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:12.967243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:13.074671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a9067-1605-46ac-8ba0-431bc67c30f7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:13.111522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e0a83-d46e-43d6-9cd5-2f5272e4680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:13.112231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0a83-d46e-43d6-9cd5-2f5272e4680a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.112525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0a83-d46e-43d6-9cd5-2f5272e4680a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.112808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0a83-d46e-43d6-9cd5-2f5272e4680a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.113566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e0a83-d46e-43d6-9cd5-2f5272e4680a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:13.113880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e0a83-d46e-43d6-9cd5-2f5272e4680a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:13.115266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e0a83-d46e-43d6-9cd5-2f5272e4680a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:13.125145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 395/784] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:13.130564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ec42b-a941-4727-938a-2e609cefea04 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:13.193883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356ce75-5ccd-4ae2-965e-2290e0521ba4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.194399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356ce75-5ccd-4ae2-965e-2290e0521ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.194694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356ce75-5ccd-4ae2-965e-2290e0521ba4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.196432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 396/785] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:13.202192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5d2f-25d8-4af6-9f14-906853736355 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.202488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5d2f-25d8-4af6-9f14-906853736355 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.202778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5d2f-25d8-4af6-9f14-906853736355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.223602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2fac5d2f-25d8-4af6-9f14-906853736355 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:13.225281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2fac5d2f-25d8-4af6-9f14-906853736355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:13.309029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 397/786] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:13.310411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e1a4c-1a1e-458b-9d82-49d8695ab5b4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.310706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e1a4c-1a1e-458b-9d82-49d8695ab5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.310988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e1a4c-1a1e-458b-9d82-49d8695ab5b4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.315470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ec42b-a941-4727-938a-2e609cefea04 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:13.316621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 398/787] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:13.322670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e99fd-b742-4994-97ba-63dbddfa66b2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.322962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e99fd-b742-4994-97ba-63dbddfa66b2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.323252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e99fd-b742-4994-97ba-63dbddfa66b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.352659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2b3e99fd-b742-4994-97ba-63dbddfa66b2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:13.353792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b3e99fd-b742-4994-97ba-63dbddfa66b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:13.365229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:13.366143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.366929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.367217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.375163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:13.384266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:13.386490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:13.416892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:13.418321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:13.420009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:13.420009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:13.420009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:13.420009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:13.420009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 390/788] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:13.420009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:13.442234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 399/789] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:13.443223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e99fd-b742-4994-97ba-63dbddfa66b2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:13.474982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:13.535594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949f97d-a85c-4aa6-9136-c363d76cfb38 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:13.573228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e99fd-b742-4994-97ba-63dbddfa66b2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:13.581257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615155a-d328-4d06-be9d-0979fdcf44a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:13.582082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615155a-d328-4d06-be9d-0979fdcf44a6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.582368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615155a-d328-4d06-be9d-0979fdcf44a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.582649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615155a-d328-4d06-be9d-0979fdcf44a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.600115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615155a-d328-4d06-be9d-0979fdcf44a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4615155a-d328-4d06-be9d-0979fdcf44a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:13.609136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:13.610299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:51:13.611348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:13.619095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 391/790] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:51:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:13.619095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615155a-d328-4d06-be9d-0979fdcf44a6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:13.619732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:13.620418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.620701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.620978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.621707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:13.622019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:13.623417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:13.641140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 400/791] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:13.642474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:13.658966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:13.729161 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615155a-d328-4d06-be9d-0979fdcf44a6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:13.736928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:13.755278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfc5aa-0ff6-4aca-b4c3-929e7ad75957 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:13.766006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dc096-fb27-42a8-ba9d-86cc0b928187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:13.767414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dc096-fb27-42a8-ba9d-86cc0b928187 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.767904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dc096-fb27-42a8-ba9d-86cc0b928187 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.768429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dc096-fb27-42a8-ba9d-86cc0b928187 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.780389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225dc096-fb27-42a8-ba9d-86cc0b928187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-225dc096-fb27-42a8-ba9d-86cc0b928187 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:13.788590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:13.789551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:13.790406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:13.797984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 392/792] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:13.797984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dc096-fb27-42a8-ba9d-86cc0b928187 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:13.802298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:13.802982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.803783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.804066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.805033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:13.805347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:13.806732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:13.821929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 401/793] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:51:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:13.821929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:13.918136 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:13.942458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dc096-fb27-42a8-ba9d-86cc0b928187 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:13.946734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93286b74-f6c2-456a-8925-a2091ec3a19d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:13.983558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:13.984257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:13.984383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.984571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.984821 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.985575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:13.985887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:13.986180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:13.986670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:13.987158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:13.987260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.000914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.001122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 402/794] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:14.001122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.001809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:14.021774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:14.023291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 30 11:51:14.024316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:14.028235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 393/795] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:14.030738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.069079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:14.133666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c742a-d81e-48b6-b4fc-c988478f09cb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.159229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151fd8-d5c5-4d5a-ada3-4b000ee44fdb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.173836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.174584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.174869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.175152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.181690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:14.195835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.200639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.216584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.217397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.217679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.217957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:14.230587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:14.231537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:14.232472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:14.232472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:14.232472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:14.232472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:14.238521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 403/796] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:14.238521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.238751 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.238751 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:14.260841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:14.261791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:14.262685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:14.262685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:14.262685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:14.262685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:14.270442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 394/797] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:14.271711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.362274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cc8c0-4e09-432b-ba62-2f85d7d4a58f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.364191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:14.390744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56f998-364f-4c60-897f-b75ddf3dd3fd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.399294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.399977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.400306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.400591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.401855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:14.401855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.403186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.427463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 404/798] 10.222.0.32 () {56 vars in 1135 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:14.428644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.433948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.434894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.435417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.435907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.436896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:14.437521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.439116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.457991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 395/799] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:14.459352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.477938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:14.536275 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfb17c-fb0c-4392-9c08-cccb8ccc14cc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.572214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a9d0f-45d5-4d66-956c-ecc267e9a720 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.577828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.578510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.578795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.579069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.580207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:14.580207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.581721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.595083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 405/800] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:14.595945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.611500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66703f8f-a7fa-4f7a-80b3-cade2686ec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.612381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66703f8f-a7fa-4f7a-80b3-cade2686ec3b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.612671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66703f8f-a7fa-4f7a-80b3-cade2686ec3b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.612951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66703f8f-a7fa-4f7a-80b3-cade2686ec3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.619828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:14.624802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66703f8f-a7fa-4f7a-80b3-cade2686ec3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.626257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66703f8f-a7fa-4f7a-80b3-cade2686ec3b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-66703f8f-a7fa-4f7a-80b3-cade2686ec3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:14.643920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:14.644875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:14.645773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:14.645773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:14.645773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:14.645773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:14.645773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 396/801] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:14.646495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba87e79-0fe5-4821-b3bf-9eeaf6b19748 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.646737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba87e79-0fe5-4821-b3bf-9eeaf6b19748 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.647601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba87e79-0fe5-4821-b3bf-9eeaf6b19748 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.649111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 397/802] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:14.650195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9c8b-c73c-4d3d-bf0c-b150cb26bf3c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.650813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9c8b-c73c-4d3d-bf0c-b150cb26bf3c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.650942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de9c8b-c73c-4d3d-bf0c-b150cb26bf3c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.652102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 398/803] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:14.652888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de9c8b-c73c-4d3d-bf0c-b150cb26bf3c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.707939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1cc75c-2471-4992-8ec7-8be43632a0e1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.715523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:14.742992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.743801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.744088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.744400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.751964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.753450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.757536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de9c8b-c73c-4d3d-bf0c-b150cb26bf3c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:14.769589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:14.770717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 406/804] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:14.771656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.796952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c19088-ab77-4731-9b35-433d10ea32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.797806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c19088-ab77-4731-9b35-433d10ea32c1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.798034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c19088-ab77-4731-9b35-433d10ea32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.798309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c19088-ab77-4731-9b35-433d10ea32c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.805544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c19088-ab77-4731-9b35-433d10ea32c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.806972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c19088-ab77-4731-9b35-433d10ea32c1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d6c19088-ab77-4731-9b35-433d10ea32c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:14.821306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:14.822269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:14.823218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 399/805] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:14.823838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4b503-0f74-4669-b10f-1e231fed7c97 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.824416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4b503-0f74-4669-b10f-1e231fed7c97 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.824416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4b503-0f74-4669-b10f-1e231fed7c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.842125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f3f4b503-0f74-4669-b10f-1e231fed7c97 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:14.843319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f3f4b503-0f74-4669-b10f-1e231fed7c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:14.876389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5495cd-4ad8-45d3-9082-4b70133a7949 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:14.890544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:14.911815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 400/806] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:14.911990 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:14.912577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.912865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.913218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.913319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffd457-71f2-4b88-9cca-417295a491a1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:14.913722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffd457-71f2-4b88-9cca-417295a491a1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:14.913950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:14.914041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffd457-71f2-4b88-9cca-417295a491a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:14.914330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:14.915705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:14.928763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 407/807] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:14.929604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:14.934922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f2ffd457-71f2-4b88-9cca-417295a491a1 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:14.936207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f2ffd457-71f2-4b88-9cca-417295a491a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:14.957619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.004620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 401/808] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:15.005403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ffd457-71f2-4b88-9cca-417295a491a1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.034150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7bdae-e7c0-4f33-940d-ec6b26f3adac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.068244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.069413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.069698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.070010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.071020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:15.071338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.072816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.090873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 408/809] 10.222.0.32 () {56 vars in 1149 bytes} [Mon May 30 11:51:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:15.091701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.120079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ffd457-71f2-4b88-9cca-417295a491a1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.130729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.156116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.156813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.157092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.157366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.158196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:15.158500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.159923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.176854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.177509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 402/810] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:15.178485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.195558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa757c-6ca3-4649-ad42-a0c562eabfac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.229939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.230735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.231017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.231292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.237941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.239381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:15.254231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:15.255426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 409/811] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:15.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.283974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6807b4a-6243-40ca-808a-d1f88e12a1d8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.320528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.321345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.321631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.321907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.322562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.323931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:15.336254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:15.337393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:15.343434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 403/812] 10.222.0.32 () {56 vars in 1129 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:15.344494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.358107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3b90b-8b11-4ef4-9c2c-3ac0089de42a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.375650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.392391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.393191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.393473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.393746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] PATH_INFO: `/v3/services/network` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.394399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.395774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:15.407914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:15.408944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:15.409784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 410/813] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/services/network => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:15.410588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.421624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.449665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2565b-984a-4fcb-a737-3c46f7645a7a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.486782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.487585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.487870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.488146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.498951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.500425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.515994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe843e2b-2c41-426d-a214-940092198b7f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:15.517678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:15.518698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:15.519705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:15.519705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:15.519705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:15.519705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:15.519705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 404/814] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:15.520054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.555897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3499b-e246-47d3-9cb3-244d82b39f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.556586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3499b-e246-47d3-9cb3-244d82b39f89 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.556877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3499b-e246-47d3-9cb3-244d82b39f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.557161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3499b-e246-47d3-9cb3-244d82b39f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.612080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.629300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a279d-58fd-47fa-8aa6-e40680590882 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.634077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3499b-e246-47d3-9cb3-244d82b39f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.635564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3499b-e246-47d3-9cb3-244d82b39f89 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.646688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.666594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.667264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.667539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.667815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.668647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:15.668955 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.670391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.682784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 405/815] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:15.683437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.743763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 411/816] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:15.744660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3499b-e246-47d3-9cb3-244d82b39f89 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.784486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cac9b-c464-401b-9a1c-f15829c67e4d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.817915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.818584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.819500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.819794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.820540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:15.820851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.822225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.826667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 406/817] 10.222.0.32 () {56 vars in 1124 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:15.827453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.839038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.850527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3499b-e246-47d3-9cb3-244d82b39f89 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.860893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:15.888501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.889473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.889946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.890416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.891659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] RBAC: Adding query filter params (name=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:15.892521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.894838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.907141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 412/818] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:15.907141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:15.939045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a0a64-2f97-4f1d-b13e-f84442420da9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:15.974866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:15.975971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:15.976297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:15.976579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:15.977302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:15.977612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:15.978982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:15.990234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 407/819] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:15.991931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.014286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939e199-d1d3-4f64-a518-4f7197c53f23 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.048481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61bce7-5ca0-4d95-a0f1-f0b3918049db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.049291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61bce7-5ca0-4d95-a0f1-f0b3918049db None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.049579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61bce7-5ca0-4d95-a0f1-f0b3918049db None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.049860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61bce7-5ca0-4d95-a0f1-f0b3918049db None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/e0cb45c1eb6040548634022673a51e56/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.052337 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61bce7-5ca0-4d95-a0f1-f0b3918049db None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=e0cb45c1eb6040548634022673a51e56, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.054301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61bce7-5ca0-4d95-a0f1-f0b3918049db None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.063005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:16.067477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 413/820] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:51:15 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/e0cb45c1eb6040548634022673a51e56/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:51:16.068430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61bce7-5ca0-4d95-a0f1-f0b3918049db None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.102790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c115a4f-47ef-41a8-b166-f9dd2e3b4260 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.127825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:16.162150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.163173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.163404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.163646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.175745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.177275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:16.197544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:16.198777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:16.199785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:16.199785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:16.199785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:16.199785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:16.199785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 408/821] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:16.199785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.224004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61bce7-5ca0-4d95-a0f1-f0b3918049db None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.269539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.270217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.270503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.270807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.271516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:16.271901 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.273324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.281209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 414/822] 10.222.0.32 () {56 vars in 1124 bytes} [Mon May 30 11:51:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:16.282317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.321212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e0434-0a0d-4fc8-a4fc-1ea4d5409428 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.345234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:16.361593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.362488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.362785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.363070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.363795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:16.364115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.365533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.370853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 409/823] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:16.376074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.391170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:16.398778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b41e2f-bb6d-4672-b49d-48c35df7bf1a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.435931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.436758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.437046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.437323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.445537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.446984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:16.462248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:16.463672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 415/824] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:16.464637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.495344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a1c9-2173-4b7e-baa0-ee1a8eb79ec2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.533640 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.534315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.534607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.534887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.535612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:16.535927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.537331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.550064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 410/825] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:16.550880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.585782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0390c9-6855-4035-91ec-ce23d37ce8a8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.605529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:16.624545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0f55d-2d0f-4709-a7f4-9b862e71fd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.625219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0f55d-2d0f-4709-a7f4-9b862e71fd18 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.625507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0f55d-2d0f-4709-a7f4-9b862e71fd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.625814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0f55d-2d0f-4709-a7f4-9b862e71fd18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.626541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0f55d-2d0f-4709-a7f4-9b862e71fd18 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.627925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0f55d-2d0f-4709-a7f4-9b862e71fd18 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.651917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 416/826] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:51:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:16.657886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0f55d-2d0f-4709-a7f4-9b862e71fd18 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.672112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aadc8f-97ec-4319-b13d-21f5b1b6ab4e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.687426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:16.722865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9c5c-f852-4576-9881-1c2e8b18214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.723703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9c5c-f852-4576-9881-1c2e8b18214b None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.724068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9c5c-f852-4576-9881-1c2e8b18214b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.724465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9c5c-f852-4576-9881-1c2e8b18214b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.725368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9c5c-f852-4576-9881-1c2e8b18214b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.726932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9c5c-f852-4576-9881-1c2e8b18214b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.751727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 411/827] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:51:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:16.753038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9c5c-f852-4576-9881-1c2e8b18214b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.809569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0f55d-2d0f-4709-a7f4-9b862e71fd18 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.858298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.859307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.859881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.860374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.861545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:16.862072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.863878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.864382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:16.882925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 417/828] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:16.883653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9c5c-f852-4576-9881-1c2e8b18214b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:16.899335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:16.936944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:16.937789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:16.938081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:16.938367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:16.959987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:16.963317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:16.983063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:16.984047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:16.984994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 412/829] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:16.990571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:17.022583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:17.064822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1359-c312-416f-935a-cceff775a23f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:17.109059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155fab7-4fdb-4e32-8d9f-fd3940661d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:17.109666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673fa8b-ce9f-487d-b436-f241b76d22fc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:17.110535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155fab7-4fdb-4e32-8d9f-fd3940661d81 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:17.110820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155fab7-4fdb-4e32-8d9f-fd3940661d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:17.111106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155fab7-4fdb-4e32-8d9f-fd3940661d81 None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/d087549c76ae4c3896c7c8c6b5a74363/roles/126791116ac8403986eead13c8c6dd07` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:17.128878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155fab7-4fdb-4e32-8d9f-fd3940661d81 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=d087549c76ae4c3896c7c8c6b5a74363, role_id=126791116ac8403986eead13c8c6dd07)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:17.130450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155fab7-4fdb-4e32-8d9f-fd3940661d81 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:17.150643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 418/830] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:51:16 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/d087549c76ae4c3896c7c8c6b5a74363/roles/126791116ac8403986eead13c8c6dd07 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:51:17.157346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0163e69-b448-4cc6-8bf7-92005268475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:17.158062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163e69-b448-4cc6-8bf7-92005268475d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:17.158393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163e69-b448-4cc6-8bf7-92005268475d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:17.158677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0163e69-b448-4cc6-8bf7-92005268475d None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:17.159616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0163e69-b448-4cc6-8bf7-92005268475d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:17.159932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0163e69-b448-4cc6-8bf7-92005268475d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:17.161617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0163e69-b448-4cc6-8bf7-92005268475d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:17.176774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 413/831] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:17.182452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155fab7-4fdb-4e32-8d9f-fd3940661d81 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:17.305083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155fab7-4fdb-4e32-8d9f-fd3940661d81 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:17.326442 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:17.347354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f91c6-1485-47e6-b7d2-647fd9ded06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:17.348037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f91c6-1485-47e6-b7d2-647fd9ded06a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:17.348367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f91c6-1485-47e6-b7d2-647fd9ded06a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:17.348644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f91c6-1485-47e6-b7d2-647fd9ded06a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:17.349496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f91c6-1485-47e6-b7d2-647fd9ded06a None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=70dce2f0284847ab8f07f55d700d0f60) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:17.349814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f91c6-1485-47e6-b7d2-647fd9ded06a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:17.352100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f91c6-1485-47e6-b7d2-647fd9ded06a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:17.359459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 419/832] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:17 2022] GET /identity/v3/role_assignments?user.id=70dce2f0284847ab8f07f55d700d0f60&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=4d21498572b14478a996ac538be5382e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:18.427512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a20d60-b092-43ed-93a3-7c4f132c34ea None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:18.428102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a20d60-b092-43ed-93a3-7c4f132c34ea None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:18.428236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a20d60-b092-43ed-93a3-7c4f132c34ea None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:18.429941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 414/833] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:18.435320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2dfa9-51bc-4565-8525-3f4747f18599 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:18.435730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2dfa9-51bc-4565-8525-3f4747f18599 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:18.435917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2dfa9-51bc-4565-8525-3f4747f18599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:18.453671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-02c2dfa9-51bc-4565-8525-3f4747f18599 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:18.454869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02c2dfa9-51bc-4565-8525-3f4747f18599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:18.527761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 420/834] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:18.551893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a20d60-b092-43ed-93a3-7c4f132c34ea None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:18.581469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:18.730927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a20d60-b092-43ed-93a3-7c4f132c34ea None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:18.789391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa96538-2940-4dec-baf9-c9ca2cf34456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:18.791262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa96538-2940-4dec-baf9-c9ca2cf34456 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:18.792056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa96538-2940-4dec-baf9-c9ca2cf34456 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:18.792843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa96538-2940-4dec-baf9-c9ca2cf34456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:18.812739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa96538-2940-4dec-baf9-c9ca2cf34456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4aa96538-2940-4dec-baf9-c9ca2cf34456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:18.829423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:18.831232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:18.832894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:18.837692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 415/835] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:18.845501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2dfa9-51bc-4565-8525-3f4747f18599 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:18.953919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2dfa9-51bc-4565-8525-3f4747f18599 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:18.968218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:18.993270 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:18.994264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:18.994734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:18.995214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:18.996188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:18.996699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:18.998257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:19.013620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 421/836] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:19.019604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa96538-2940-4dec-baf9-c9ca2cf34456 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:19.105028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:19.129956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa96538-2940-4dec-baf9-c9ca2cf34456 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:19.167366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c10bdb-9157-40f1-9141-126f2a8e3592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:19.168189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c10bdb-9157-40f1-9141-126f2a8e3592 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.168488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c10bdb-9157-40f1-9141-126f2a8e3592 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.168769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c10bdb-9157-40f1-9141-126f2a8e3592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.180033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c10bdb-9157-40f1-9141-126f2a8e3592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:19.181535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c10bdb-9157-40f1-9141-126f2a8e3592 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b1c10bdb-9157-40f1-9141-126f2a8e3592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:19.199706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:19.200690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:19.201566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:19.201566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:19.201566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:19.201566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:19.205577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 416/837] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:19.206954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:19.244246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da949583-660c-4498-90ed-3b3bf498f36e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.244553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da949583-660c-4498-90ed-3b3bf498f36e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.244732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da949583-660c-4498-90ed-3b3bf498f36e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.246359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 417/838] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:19.251363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a74c5-5e1a-44aa-bd5c-3e1c574dd3c8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.251652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a74c5-5e1a-44aa-bd5c-3e1c574dd3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.251932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a74c5-5e1a-44aa-bd5c-3e1c574dd3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.269398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-be6a74c5-5e1a-44aa-bd5c-3e1c574dd3c8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:19.270600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-be6a74c5-5e1a-44aa-bd5c-3e1c574dd3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:19.328423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee5046-7b65-4e59-9316-c9f7d1b96578 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:19.354210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 418/839] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:19.368785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:19.369464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.369758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.370452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.371190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:19.371344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a74c5-5e1a-44aa-bd5c-3e1c574dd3c8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:19.371497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:19.372911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:19.391405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 422/840] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:19.392984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:19.429381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:19.435205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:19.481903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a74c5-5e1a-44aa-bd5c-3e1c574dd3c8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:19.502901 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be451c33-d61e-445b-a0b6-a934dd5d7861 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:19.519170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:19.519992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.520328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.520614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.521280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:19.523217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:19.536347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:19.538260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:19.540266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:19.544192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 419/841] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:19.544621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:19.545545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.545903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:19.546025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.546370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.554655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:19.556625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:19.573182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:19.574423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 30 11:51:19.575480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:19.580411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 423/842] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:19.582036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:19.662123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8d754-07b1-4eaa-8fc1-95c43e281370 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:19.685951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:19.688419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:19.695296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc6b27-6625-43a7-9b22-4c1675e1d00e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:19.702444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:19.703568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.704087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.704773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.705757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:19.706316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:19.708036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:19.718844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 420/843] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:19.720263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:19.743102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:19.743823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.744109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.744418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.745273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:19.745585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:19.747034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:19.775116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 424/844] 10.222.0.32 () {56 vars in 1135 bytes} [Mon May 30 11:51:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:19.781581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:19.850840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92db2dd-1a8b-4f89-8ef1-2ae8243cce0c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:19.895079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692cabe-c13e-491d-9922-da40553ac419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:19.896359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692cabe-c13e-491d-9922-da40553ac419 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.897441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692cabe-c13e-491d-9922-da40553ac419 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.897994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692cabe-c13e-491d-9922-da40553ac419 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.899266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692cabe-c13e-491d-9922-da40553ac419 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:19.899741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692cabe-c13e-491d-9922-da40553ac419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:19.901912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f692cabe-c13e-491d-9922-da40553ac419 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:19.903006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d0808-801a-420b-a5a9-d773b0084c65 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:19.916896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 421/845] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:19.957629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df616c24-43c3-4b24-9b22-24984f1e4692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:19.958313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df616c24-43c3-4b24-9b22-24984f1e4692 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:19.958601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df616c24-43c3-4b24-9b22-24984f1e4692 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:19.958883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df616c24-43c3-4b24-9b22-24984f1e4692 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:19.984552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:20.042752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df616c24-43c3-4b24-9b22-24984f1e4692 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:20.044268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df616c24-43c3-4b24-9b22-24984f1e4692 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:20.069267 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc395-3407-4ec6-9ac0-bedb3958b27d None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.069555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc395-3407-4ec6-9ac0-bedb3958b27d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.069835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc395-3407-4ec6-9ac0-bedb3958b27d None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.071538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 422/846] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:20.077219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d7561-14b1-4f71-a63f-af20282d54a3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.077497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d7561-14b1-4f71-a63f-af20282d54a3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.077779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d7561-14b1-4f71-a63f-af20282d54a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.098546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c48d7561-14b1-4f71-a63f-af20282d54a3 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:20.099768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c48d7561-14b1-4f71-a63f-af20282d54a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:20.165687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:20.175480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 425/847] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:20.181290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df616c24-43c3-4b24-9b22-24984f1e4692 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:20.191983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 423/848] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:20.220742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d7561-14b1-4f71-a63f-af20282d54a3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:20.239173 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:20.296149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df616c24-43c3-4b24-9b22-24984f1e4692 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:20.332991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018700e-d590-43b1-9f9b-b01863731560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:20.333796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018700e-d590-43b1-9f9b-b01863731560 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.334078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018700e-d590-43b1-9f9b-b01863731560 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.334479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018700e-d590-43b1-9f9b-b01863731560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.346547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018700e-d590-43b1-9f9b-b01863731560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:20.348027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018700e-d590-43b1-9f9b-b01863731560 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:20.351760 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d7561-14b1-4f71-a63f-af20282d54a3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9018700e-d590-43b1-9f9b-b01863731560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:20.367719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:20.368823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:20.369754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:20.369754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:20.369754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:20.369754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:20.369754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 426/849] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:20.370368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10de2-9685-4157-9a67-cad06345e6a7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.370660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10de2-9685-4157-9a67-cad06345e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.370937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10de2-9685-4157-9a67-cad06345e6a7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.376186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 427/850] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:20.376186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10de2-9685-4157-9a67-cad06345e6a7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:20.398499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c583f-17de-420b-938b-7b0ce54fee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:20.399315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c583f-17de-420b-938b-7b0ce54fee25 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.400072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c583f-17de-420b-938b-7b0ce54fee25 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.400396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c583f-17de-420b-938b-7b0ce54fee25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.411795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c583f-17de-420b-938b-7b0ce54fee25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-693c583f-17de-420b-938b-7b0ce54fee25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:20.420263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:20.421231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:20.422071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 424/851] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:20.423174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccf1bd-734d-4882-b9a1-7991e038bc83 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.423465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccf1bd-734d-4882-b9a1-7991e038bc83 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.423753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccf1bd-734d-4882-b9a1-7991e038bc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.443762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-09ccf1bd-734d-4882-b9a1-7991e038bc83 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:20.445109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-09ccf1bd-734d-4882-b9a1-7991e038bc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:20.483322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:20.490014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10de2-9685-4157-9a67-cad06345e6a7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:20.492799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:20.522329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 425/852] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:20.523266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccf1bd-734d-4882-b9a1-7991e038bc83 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:20.534549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:20.535958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.536525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.537022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.538144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:20.538675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:20.540414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:20.556371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 428/853] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:20.556371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:20.653769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccf1bd-734d-4882-b9a1-7991e038bc83 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:20.665898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa9278-a413-4d4b-841b-5d15a8403f06 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:20.698223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:20.699171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.699458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.699739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.700537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:20.700853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:20.702226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:20.702692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:20.703508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.703795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.704075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.704764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:20.706125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:20.717187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 426/854] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:20.717187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:20.719231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:20.720148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:20.721030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:20.725321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 429/855] 10.222.0.32 () {56 vars in 1132 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:20.726499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:20.750262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:20.759715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:20.829670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895873bc-e653-4891-bfb4-07c18ec07290 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:20.836689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fb06a-c27e-456e-9949-db8109055fcc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:20.867698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:20.868528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.868807 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.869250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.874066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:20.874864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:20.875152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:20.875425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:20.876056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:20.877518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:20.888287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:20.889731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:20.893073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:20.893949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 427/856] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:20.895365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:20.913760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:20.914692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:20.915606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:20.915606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:20.915606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:20.915606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:20.915606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 430/857] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:20.916695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:21.002615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:21.003851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:21.014564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8525a-0e8d-442b-8d76-7ba9c35db2f6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:21.031191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0e1b4-e23a-4553-bd6e-54b26185b7cd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:21.057385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:21.057877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:21.058096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:21.058369 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:21.059190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:21.059498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:21.060907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:21.071221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 428/858] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:21.071221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:21.072649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:21.073341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:21.073638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:21.073928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:21.074732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:21.075055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:21.076574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:21.090170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 431/859] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:21.095542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:21.184510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd63eec-70db-4f4b-bf31-0c7e6e0fdf82 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:21.212236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80b06a-0026-4825-85dd-88c1f63a1397 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:21.223529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:21.223951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:21.224264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:21.224543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:21.225332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:21.225623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:21.227000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:21.243649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 429/860] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:21.243649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:21.259857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:21.260486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:21.260729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:21.261011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:21.261735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:21.262064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:21.263506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:21.263995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:21.280194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 432/861] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:21.280194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:21.286364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:21.361310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04e812-0e3e-4620-bae8-0cf8a0f713be None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:21.413540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:21.414216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:21.414506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:21.414791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:21.415637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=e0cb45c1eb6040548634022673a51e56) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:21.415957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:21.417478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:21.424669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 430/862] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:21 2022] GET /identity/v3/role_assignments?user.id=e0cb45c1eb6040548634022673a51e56&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=4d21498572b14478a996ac538be5382e => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:21.428898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054f658-f19e-4150-b976-322e801f39e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:21.471633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:21.472517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:21.472800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:21.473072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:21.482069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:21.483560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:21.501393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:21.502423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:21.503580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 433/863] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:51:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:21.511255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:21.620324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:21.643004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762787ae-bf04-4629-bbf1-79b20d6ae461 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:21.681172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:21.681860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:21.682146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:21.682426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:21.683280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:21.683596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:21.685063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:21.707181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 431/864] 10.222.0.32 () {56 vars in 1149 bytes} [Mon May 30 11:51:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:21.713809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:21.792059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:21.834967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980b8f5-26a9-4eca-9176-fd248728c40f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:21.869753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680756e-96ae-43aa-9f7a-2bffbd076ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:21.870473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680756e-96ae-43aa-9f7a-2bffbd076ebf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:21.870702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680756e-96ae-43aa-9f7a-2bffbd076ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:21.870978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680756e-96ae-43aa-9f7a-2bffbd076ebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:21.946359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680756e-96ae-43aa-9f7a-2bffbd076ebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:21.947849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680756e-96ae-43aa-9f7a-2bffbd076ebf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:22.024569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:22.065713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 434/865] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:22.072311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:22.188973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d8298-9b64-4e43-a40f-a08e1d8540c8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:22.225358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfe5ae-9ddf-471c-924d-b027490e274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:22.226183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfe5ae-9ddf-471c-924d-b027490e274d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:22.226472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfe5ae-9ddf-471c-924d-b027490e274d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:22.226755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfe5ae-9ddf-471c-924d-b027490e274d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:22.236798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:22.238821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfe5ae-9ddf-471c-924d-b027490e274d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:22.240558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfe5ae-9ddf-471c-924d-b027490e274d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3fcfe5ae-9ddf-471c-924d-b027490e274d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:22.259517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:22.260489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:22.261598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:22.261598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:22.261598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:22.261598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:22.261598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 432/866] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:22.268176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680756e-96ae-43aa-9f7a-2bffbd076ebf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:22.306936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe45ba-9961-4f2a-8b1c-c8136a1d9a1a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:22.307248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe45ba-9961-4f2a-8b1c-c8136a1d9a1a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:22.307506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe45ba-9961-4f2a-8b1c-c8136a1d9a1a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:22.309744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 433/867] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:22.317540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ec17-5373-4f1b-8c0a-8f914ca22cd2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:22.317964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ec17-5373-4f1b-8c0a-8f914ca22cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:22.318353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150ec17-5373-4f1b-8c0a-8f914ca22cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:22.339360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3150ec17-5373-4f1b-8c0a-8f914ca22cd2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:22.341201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3150ec17-5373-4f1b-8c0a-8f914ca22cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:22.385556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680756e-96ae-43aa-9f7a-2bffbd076ebf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:22.423607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 434/868] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:22.426910 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:22.427592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:22.427971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:22.428304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:22.429043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:22.429356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:22.430741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:22.444361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 435/869] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:22.447395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ec17-5373-4f1b-8c0a-8f914ca22cd2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:22.449678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:22.480925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:22.563507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150ec17-5373-4f1b-8c0a-8f914ca22cd2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:22.584098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404565d-b76d-4ae7-80bd-f79d6fab62a9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:22.588258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:22.606967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28b988-1575-4eb7-922e-d24f95a254f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:22.607913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28b988-1575-4eb7-922e-d24f95a254f1 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:22.608491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28b988-1575-4eb7-922e-d24f95a254f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:22.608920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28b988-1575-4eb7-922e-d24f95a254f1 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:22.609977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28b988-1575-4eb7-922e-d24f95a254f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:22.611588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28b988-1575-4eb7-922e-d24f95a254f1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:22.628887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:22.629749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:22.630036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:22.630315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:22.634179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 435/870] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:51:22 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:51:22.644222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:22.644222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:22.661159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:22.662857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:22.663943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:22.663943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:22.663943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:22.663943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:22.663943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:22.663943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:22.663943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:22.663943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:22.664591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 436/871] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:22.670460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28b988-1575-4eb7-922e-d24f95a254f1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:22.809665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28b988-1575-4eb7-922e-d24f95a254f1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:22.848920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca27909-d803-42f0-8420-92e40d9d6a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:22.849657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca27909-d803-42f0-8420-92e40d9d6a99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:22.850048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca27909-d803-42f0-8420-92e40d9d6a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:22.850400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca27909-d803-42f0-8420-92e40d9d6a99 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:22.851353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca27909-d803-42f0-8420-92e40d9d6a99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:22.851675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca27909-d803-42f0-8420-92e40d9d6a99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:22.853131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca27909-d803-42f0-8420-92e40d9d6a99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:22.866843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 436/872] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:22.872742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:22.968742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:22.984981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a69860-8d8e-48f5-a240-afa09dae4272 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:23.024359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2e85-c033-4d84-8d0b-8db881a27480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:23.025425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2e85-c033-4d84-8d0b-8db881a27480 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:23.025969 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2e85-c033-4d84-8d0b-8db881a27480 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:23.026469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2e85-c033-4d84-8d0b-8db881a27480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:23.027511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2e85-c033-4d84-8d0b-8db881a27480 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=d087549c76ae4c3896c7c8c6b5a74363) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:23.028031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2e85-c033-4d84-8d0b-8db881a27480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:23.029690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd2e85-c033-4d84-8d0b-8db881a27480 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:23.040336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 437/873] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:22 2022] GET /identity/v3/role_assignments?user.id=d087549c76ae4c3896c7c8c6b5a74363&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:24.148575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea976ba2-7576-40b3-8583-6fc57d160fd7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:24.149620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea976ba2-7576-40b3-8583-6fc57d160fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:24.150137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea976ba2-7576-40b3-8583-6fc57d160fd7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:24.152020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 437/874] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:24.160412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96915e6f-1db7-4897-8902-b7f3534f6e3d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:24.160946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96915e6f-1db7-4897-8902-b7f3534f6e3d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:24.161461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96915e6f-1db7-4897-8902-b7f3534f6e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:24.189170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-96915e6f-1db7-4897-8902-b7f3534f6e3d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:24.191176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-96915e6f-1db7-4897-8902-b7f3534f6e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:24.275020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0cda9-bbfd-4c6a-bf09-263478a9443e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:24.275520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0cda9-bbfd-4c6a-bf09-263478a9443e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:24.276369 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0cda9-bbfd-4c6a-bf09-263478a9443e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:24.280477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 438/875] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:24.286414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a1534-b45a-4cd7-8df5-7b6bb61735d8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:24.286817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a1534-b45a-4cd7-8df5-7b6bb61735d8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:24.287281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a1534-b45a-4cd7-8df5-7b6bb61735d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:24.303546 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 438/876] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:24.308783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-491a1534-b45a-4cd7-8df5-7b6bb61735d8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:24.310173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-491a1534-b45a-4cd7-8df5-7b6bb61735d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:24.329140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96915e6f-1db7-4897-8902-b7f3534f6e3d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:24.349763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:24.406210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 439/877] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:24 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:24.429069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:24.429956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a1534-b45a-4cd7-8df5-7b6bb61735d8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:24.456831 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96915e6f-1db7-4897-8902-b7f3534f6e3d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:24.501849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b25da-894b-4b50-898e-238b77c4cd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:24.502532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b25da-894b-4b50-898e-238b77c4cd91 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:24.502820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b25da-894b-4b50-898e-238b77c4cd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:24.503102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b25da-894b-4b50-898e-238b77c4cd91 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:24.504087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b25da-894b-4b50-898e-238b77c4cd91 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:24.506225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b25da-894b-4b50-898e-238b77c4cd91 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:24.527073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 439/878] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:51:24 2022] POST /identity/v3/services => generated 269 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:24.552551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a1534-b45a-4cd7-8df5-7b6bb61735d8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:24.605770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:24.606994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:24.607485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:24.607965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:24.609069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:24.610974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:24.625588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:24.626670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:24.627516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:24.630706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 440/879] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:24.641370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b25da-894b-4b50-898e-238b77c4cd91 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:24.759504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b25da-894b-4b50-898e-238b77c4cd91 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:24.797591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a70474-685e-491e-9bde-5b85ece5ffbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:24.798268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a70474-685e-491e-9bde-5b85ece5ffbd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:24.798558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a70474-685e-491e-9bde-5b85ece5ffbd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:24.798841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a70474-685e-491e-9bde-5b85ece5ffbd None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:24.799555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a70474-685e-491e-9bde-5b85ece5ffbd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:24.799871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a70474-685e-491e-9bde-5b85ece5ffbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:24.801289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a70474-685e-491e-9bde-5b85ece5ffbd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:24.807068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 440/880] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:24.812349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:24.843550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:24.920794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1877b-11f6-4c9b-b2dd-fd6a9c0f5676 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:24.956623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5c5b5-0aed-48fe-b87e-0766c42fe972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:24.957295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5c5b5-0aed-48fe-b87e-0766c42fe972 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:24.957577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5c5b5-0aed-48fe-b87e-0766c42fe972 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:24.957850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5c5b5-0aed-48fe-b87e-0766c42fe972 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:24.958558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5c5b5-0aed-48fe-b87e-0766c42fe972 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:24.958862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5c5b5-0aed-48fe-b87e-0766c42fe972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:24.960916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5c5b5-0aed-48fe-b87e-0766c42fe972 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:24.966166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 441/881] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:25.023027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae47281-497b-460d-be72-725e4c9ac2a2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.023404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae47281-497b-460d-be72-725e4c9ac2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.023689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae47281-497b-460d-be72-725e4c9ac2a2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.025484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 441/882] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:25.031531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456631b-4203-444c-b26e-cafb52d046cd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.031833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456631b-4203-444c-b26e-cafb52d046cd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.032099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456631b-4203-444c-b26e-cafb52d046cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.051871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b456631b-4203-444c-b26e-cafb52d046cd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:25.053483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b456631b-4203-444c-b26e-cafb52d046cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:25.132333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 442/883] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:25.155982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae47281-497b-460d-be72-725e4c9ac2a2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:25.159327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:25.272550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae47281-497b-460d-be72-725e4c9ac2a2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:25.307322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0026d-9c0d-4b26-b447-4b3c1584022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:25.308195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0026d-9c0d-4b26-b447-4b3c1584022a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.308487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0026d-9c0d-4b26-b447-4b3c1584022a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.308765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0026d-9c0d-4b26-b447-4b3c1584022a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.309422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0026d-9c0d-4b26-b447-4b3c1584022a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:25.310814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0026d-9c0d-4b26-b447-4b3c1584022a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bbf0026d-9c0d-4b26-b447-4b3c1584022a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:25.323475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:25.324582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:25.325414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 442/884] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:25.330951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456631b-4203-444c-b26e-cafb52d046cd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:25.353875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:25.437231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456631b-4203-444c-b26e-cafb52d046cd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:25.467707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3a9b-7701-4ae1-b976-e7b7ff5d0feb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.468015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3a9b-7701-4ae1-b976-e7b7ff5d0feb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.468333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa3a9b-7701-4ae1-b976-e7b7ff5d0feb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.469970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 443/885] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:25.475069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d034e10-1c69-4883-9cf1-a0efdc95f053 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.475427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d034e10-1c69-4883-9cf1-a0efdc95f053 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.475687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d034e10-1c69-4883-9cf1-a0efdc95f053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.479675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:25.480438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.480734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.481017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.481745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:25.482061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:25.483450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:25.488954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 443/886] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:25.494342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:25.497616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5d034e10-1c69-4883-9cf1-a0efdc95f053 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:25.499269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5d034e10-1c69-4883-9cf1-a0efdc95f053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:25.582358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:25.600373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 444/887] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:25.604464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:25.624985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746369c-35cd-4328-9752-98b9858e1e5c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:25.626077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d034e10-1c69-4883-9cf1-a0efdc95f053 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:25.672834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:25.673895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.674395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.674896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.676262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:25.676782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:25.678918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:25.686472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 444/888] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:25.691668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:25.746882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d034e10-1c69-4883-9cf1-a0efdc95f053 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:25.785608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114bc52-3b01-4d06-98a7-8c0707eab49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:25.786904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bc52-3b01-4d06-98a7-8c0707eab49c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.787968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bc52-3b01-4d06-98a7-8c0707eab49c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.788586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bc52-3b01-4d06-98a7-8c0707eab49c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.800395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114bc52-3b01-4d06-98a7-8c0707eab49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:25.804440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700baf37-5a9e-47f7-9353-b5b1810c7d8b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f114bc52-3b01-4d06-98a7-8c0707eab49c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:25.809838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:25.810794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:25.811655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:25.814697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 445/889] 10.222.0.32 () {56 vars in 1120 bytes} [Mon May 30 11:51:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:25.823668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114bc52-3b01-4d06-98a7-8c0707eab49c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:25.844376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:25.844995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:25.845393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:25.845675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:25.846592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] RBAC: Adding query filter params (interface=public, service_id=79d0c2fdf5924a63bf912844028d2a19, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:25.846908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:25.848307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:25.858928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 445/890] 10.222.0.32 () {56 vars in 1251 bytes} [Mon May 30 11:51:25 2022] GET /identity/v3/endpoints?service_id=79d0c2fdf5924a63bf912844028d2a19&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:25.860349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:25.879203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:25.902610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:25.972812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114bc52-3b01-4d06-98a7-8c0707eab49c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:26.006939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b152-5073-45fb-9375-424bc683dc15 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:26.023228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:26.024125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:26.024544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:26.024928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:26.025788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:26.026273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:26.027747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:26.041663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 446/891] 10.222.0.32 () {56 vars in 1115 bytes} [Mon May 30 11:51:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:26.042700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:26.060830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561125e9-3972-4bbe-aaac-19d1fd641b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:26.061518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561125e9-3972-4bbe-aaac-19d1fd641b54 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:26.061793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561125e9-3972-4bbe-aaac-19d1fd641b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:26.062075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561125e9-3972-4bbe-aaac-19d1fd641b54 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:26.062838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561125e9-3972-4bbe-aaac-19d1fd641b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:26.064307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561125e9-3972-4bbe-aaac-19d1fd641b54 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:26.075496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 446/892] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:51:25 2022] GET /identity/v3/services => generated 1268 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 11:51:26.170588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99643c-48e6-490e-9559-b7cdba102537 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:26.192574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:26.213417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:26.214618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:26.215041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:26.215455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:26.229787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:26.231291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:26.253696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:26.258163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:26.260610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:26.260610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:26.260610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:26.260610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:26.260610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 447/893] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:26.262453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561125e9-3972-4bbe-aaac-19d1fd641b54 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:26.353490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:26.379348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561125e9-3972-4bbe-aaac-19d1fd641b54 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:26.417181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:26.418238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:26.418739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:26.419239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:26.420212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:26.420786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:26.422373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:26.439604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 447/894] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:26.445345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:26.553749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493cc5e-75c0-4c12-9bc9-7698240884ae None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:26.589102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:26.589929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:26.590217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:26.590497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:26.599070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:26.600621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:26.616603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:26.617591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:26.618549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 448/895] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:51:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:26.623961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:26.733822 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e967f1-4d7c-43b8-a325-09c3a6a1dd94 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:26.775116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6016a5-016c-4d68-adf3-ef1f16829b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:26.775813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6016a5-016c-4d68-adf3-ef1f16829b61 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:26.776105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6016a5-016c-4d68-adf3-ef1f16829b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:26.776412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6016a5-016c-4d68-adf3-ef1f16829b61 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:26.777359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6016a5-016c-4d68-adf3-ef1f16829b61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:26.777676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6016a5-016c-4d68-adf3-ef1f16829b61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:26.779127 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6016a5-016c-4d68-adf3-ef1f16829b61 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:26.782107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:26.798862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 448/896] 10.222.0.32 () {56 vars in 1149 bytes} [Mon May 30 11:51:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:26.804905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:26.838531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:26.914920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62581e3d-2008-4ee5-a733-0cad67e51861 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:26.950730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51c586-16df-41a3-b8ee-b9a8c8f57a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:26.951397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51c586-16df-41a3-b8ee-b9a8c8f57a94 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:26.951683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51c586-16df-41a3-b8ee-b9a8c8f57a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:26.951963 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51c586-16df-41a3-b8ee-b9a8c8f57a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.027300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51c586-16df-41a3-b8ee-b9a8c8f57a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:27.028769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51c586-16df-41a3-b8ee-b9a8c8f57a94 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:27.070961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:27.111129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c31c-1067-4b7a-90e9-6df3e9b72f48 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.111458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c31c-1067-4b7a-90e9-6df3e9b72f48 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.111698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c31c-1067-4b7a-90e9-6df3e9b72f48 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.113789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 449/897] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:27.124831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c97c-2e60-4554-a44b-7f5214d2fb57 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.125553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c97c-2e60-4554-a44b-7f5214d2fb57 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.126157 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c97c-2e60-4554-a44b-7f5214d2fb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.144737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b3b4c97c-2e60-4554-a44b-7f5214d2fb57 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:27.145205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 449/898] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:26 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:27.147195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b3b4c97c-2e60-4554-a44b-7f5214d2fb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:27.154920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51c586-16df-41a3-b8ee-b9a8c8f57a94 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.223582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 450/899] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:27.254089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c97c-2e60-4554-a44b-7f5214d2fb57 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.279514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51c586-16df-41a3-b8ee-b9a8c8f57a94 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:27.310401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:27.317120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47fc1d-bc1b-4fe0-9af3-2c580906e3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:27.318018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47fc1d-bc1b-4fe0-9af3-2c580906e3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.318435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47fc1d-bc1b-4fe0-9af3-2c580906e3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.318799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47fc1d-bc1b-4fe0-9af3-2c580906e3a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.329934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47fc1d-bc1b-4fe0-9af3-2c580906e3a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:27.331498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47fc1d-bc1b-4fe0-9af3-2c580906e3a1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8e47fc1d-bc1b-4fe0-9af3-2c580906e3a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:27.351022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:27.351968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:27.352890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:27.352890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:27.352890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:27.352890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:27.352890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 450/900] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:27.354989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b04c77-3175-45e8-84c7-7fe9b8e01dae None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.355398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b04c77-3175-45e8-84c7-7fe9b8e01dae None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.355785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b04c77-3175-45e8-84c7-7fe9b8e01dae None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.357663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 451/901] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:27.358565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b04c77-3175-45e8-84c7-7fe9b8e01dae None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.375925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c97c-2e60-4554-a44b-7f5214d2fb57 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:27.403810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:27.420743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec2b5b-dc58-4b41-8723-adeba045b4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:27.421824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2b5b-dc58-4b41-8723-adeba045b4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.422516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2b5b-dc58-4b41-8723-adeba045b4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.422813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2b5b-dc58-4b41-8723-adeba045b4a3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.423861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec2b5b-dc58-4b41-8723-adeba045b4a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:27.426071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec2b5b-dc58-4b41-8723-adeba045b4a3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d4ec2b5b-dc58-4b41-8723-adeba045b4a3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:27.444464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:27.445644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:27.446486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 451/902] 10.222.0.32 () {56 vars in 1132 bytes} [Mon May 30 11:51:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:27.447872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74134-0d25-415c-bdbc-c8c9f6983c08 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.448140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74134-0d25-415c-bdbc-c8c9f6983c08 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.448450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74134-0d25-415c-bdbc-c8c9f6983c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.466593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-def74134-0d25-415c-bdbc-c8c9f6983c08 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:27.467790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-def74134-0d25-415c-bdbc-c8c9f6983c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:27.475004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b04c77-3175-45e8-84c7-7fe9b8e01dae None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:27.516196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:27.517491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.518092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.518584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.519559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:27.520064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:27.521880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:27.536650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 452/903] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:27.537755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.543564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 452/904] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:27.544612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def74134-0d25-415c-bdbc-c8c9f6983c08 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.594851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:27.659158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce093a8-9911-458a-88a5-f94c36f7edb5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:27.669202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def74134-0d25-415c-bdbc-c8c9f6983c08 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:27.687742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:27.695747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:27.696452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.696741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.697025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.697740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:27.698056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:27.699446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:27.709832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 453/905] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:27.709832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.711700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:27.712642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.713002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.713356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.720803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:27.722345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:27.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:27.737536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:27.738627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 453/906] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:27.739290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.818148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c804c-a5cd-4491-9a93-79aec95c523a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:27.826551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:27.857328 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3811ad3-3be7-4096-a8bc-5c2927c265e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:27.858004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3811ad3-3be7-4096-a8bc-5c2927c265e6 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.858294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3811ad3-3be7-4096-a8bc-5c2927c265e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.858575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3811ad3-3be7-4096-a8bc-5c2927c265e6 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.859303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3811ad3-3be7-4096-a8bc-5c2927c265e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:27.859620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86409019-db3e-4852-afef-6398ff18fa18 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:27.860728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3811ad3-3be7-4096-a8bc-5c2927c265e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:27.886381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 454/907] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:51:27 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:27.886381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3811ad3-3be7-4096-a8bc-5c2927c265e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.920269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:27.921404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:27.921919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:27.922390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:27.923317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:27.923948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:27.925600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:27.937558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 454/908] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:27.939177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:27.991744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:28.066288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3811ad3-3be7-4096-a8bc-5c2927c265e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:28.075191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899649f-290d-48ab-b668-fcfdd63f30bc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:28.113350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:28.114627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:28.114947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:28.115803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:28.118698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:28.119816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:28.122876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:28.125687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:28.126442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:28.126731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:28.127009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:28.127828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] RBAC: Adding query filter params (interface=public, service_id=092763f216c04cecbfebc813eb952681, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:28.128143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:28.129537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:28.135476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 455/909] 10.222.0.32 () {56 vars in 1251 bytes} [Mon May 30 11:51:27 2022] GET /identity/v3/endpoints?service_id=092763f216c04cecbfebc813eb952681&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:28.141389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:28.144196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 455/910] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:28.150324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:28.185258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:28.282696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09a0e8-9f07-4fd8-9c10-85e16243525d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:28.298071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3501b-1554-440b-88de-f05d76926e61 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:28.300058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:28.333396 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15481677-d82f-46e2-ac2a-7b8826941b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:28.335179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15481677-d82f-46e2-ac2a-7b8826941b20 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:28.335646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15481677-d82f-46e2-ac2a-7b8826941b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:28.336080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15481677-d82f-46e2-ac2a-7b8826941b20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:28.337377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15481677-d82f-46e2-ac2a-7b8826941b20 None admin] RBAC: Adding query filter params (role.id=8de96040db1e4e6e8c527b555d4c885a, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=d087549c76ae4c3896c7c8c6b5a74363) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:28.338399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15481677-d82f-46e2-ac2a-7b8826941b20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:28.340353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15481677-d82f-46e2-ac2a-7b8826941b20 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:28.351099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd8f78-bc30-4280-9540-23917710fe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:28.351942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 456/911] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:28 2022] GET /identity/v3/role_assignments?user.id=d087549c76ae4c3896c7c8c6b5a74363&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=8de96040db1e4e6e8c527b555d4c885a => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:28.360288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8f78-bc30-4280-9540-23917710fe11 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:28.360288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8f78-bc30-4280-9540-23917710fe11 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:28.360288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8f78-bc30-4280-9540-23917710fe11 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:28.360288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd8f78-bc30-4280-9540-23917710fe11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:28.360288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd8f78-bc30-4280-9540-23917710fe11 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:28.368306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 456/912] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:51:28 2022] GET /identity/v3/services => generated 1497 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 11:51:28.547607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88972cb2-6e08-4dec-9c36-7ebc4e5a9a42 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:28.548366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88972cb2-6e08-4dec-9c36-7ebc4e5a9a42 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:28.548366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88972cb2-6e08-4dec-9c36-7ebc4e5a9a42 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:28.549764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 457/913] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:28.556662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f66f-4b0a-4ccd-bfdb-d2ec682498d9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:28.557308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f66f-4b0a-4ccd-bfdb-d2ec682498d9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:28.557308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f66f-4b0a-4ccd-bfdb-d2ec682498d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:28.575536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bf19f66f-4b0a-4ccd-bfdb-d2ec682498d9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:28.577150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bf19f66f-4b0a-4ccd-bfdb-d2ec682498d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:28.659773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 457/914] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:28.682751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88972cb2-6e08-4dec-9c36-7ebc4e5a9a42 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:28.793181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:28.797935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88972cb2-6e08-4dec-9c36-7ebc4e5a9a42 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:28.835447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:28.836287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:28.836575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:28.836854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:28.837516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:28.838915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:28.852054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:28.853113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:28.854406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 458/915] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:28.859522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19f66f-4b0a-4ccd-bfdb-d2ec682498d9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:28.963696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:28.970984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19f66f-4b0a-4ccd-bfdb-d2ec682498d9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:29.008840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:29.009912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:29.010405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:29.010901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:29.011889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:29.012448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:29.014027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:29.020076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 458/916] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:29.029510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:29.137706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d76c4e-0d1c-4fed-9069-9b54ffc10a5b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:29.175604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862247d5-57f8-4d85-8929-1feab16302cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:29.176809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862247d5-57f8-4d85-8929-1feab16302cb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:29.177328 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862247d5-57f8-4d85-8929-1feab16302cb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:29.177852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862247d5-57f8-4d85-8929-1feab16302cb None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:29.179017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862247d5-57f8-4d85-8929-1feab16302cb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:29.179529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862247d5-57f8-4d85-8929-1feab16302cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:29.181800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862247d5-57f8-4d85-8929-1feab16302cb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:29.188990 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 459/917] 10.222.0.32 () {56 vars in 1114 bytes} [Mon May 30 11:51:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:29.195117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:29.309082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeda251-534b-402f-8a4c-e19241d42b47 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:29.346412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba47de8-ab9f-420d-afe2-6fdf438e6bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:29.347361 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba47de8-ab9f-420d-afe2-6fdf438e6bb2 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:29.347802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba47de8-ab9f-420d-afe2-6fdf438e6bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:29.348280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba47de8-ab9f-420d-afe2-6fdf438e6bb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:29.349182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba47de8-ab9f-420d-afe2-6fdf438e6bb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:29.351332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba47de8-ab9f-420d-afe2-6fdf438e6bb2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:29.372438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 459/918] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:51:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:30.333614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ea8b3-43d3-412b-ace0-fccce6e9ee60 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:30.334738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ea8b3-43d3-412b-ace0-fccce6e9ee60 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:30.334738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ea8b3-43d3-412b-ace0-fccce6e9ee60 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:30.336444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 460/919] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:30.342144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf946a-e930-4548-98df-5f54e9227750 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:30.342601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf946a-e930-4548-98df-5f54e9227750 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:30.342698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf946a-e930-4548-98df-5f54e9227750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:30.360390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8ebf946a-e930-4548-98df-5f54e9227750 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:30.361580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8ebf946a-e930-4548-98df-5f54e9227750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:30.377965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:30.465401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 460/920] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:30.494759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ea8b3-43d3-412b-ace0-fccce6e9ee60 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:30.516537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:30.611989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ea8b3-43d3-412b-ace0-fccce6e9ee60 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:30.647895 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602f72f-92fc-4d75-9878-769439cbb6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:30.648808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602f72f-92fc-4d75-9878-769439cbb6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:30.649094 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602f72f-92fc-4d75-9878-769439cbb6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:30.649367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602f72f-92fc-4d75-9878-769439cbb6b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:30.650080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602f72f-92fc-4d75-9878-769439cbb6b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:30.651476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602f72f-92fc-4d75-9878-769439cbb6b5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7602f72f-92fc-4d75-9878-769439cbb6b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:30.664622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:30.665860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:30.666867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 461/921] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:30.672198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf946a-e930-4548-98df-5f54e9227750 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:30.790864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf946a-e930-4548-98df-5f54e9227750 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:30.793629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae2b44-d81b-42b4-8e0b-b263dfd4cbde None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:30.793921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae2b44-d81b-42b4-8e0b-b263dfd4cbde None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:30.794203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae2b44-d81b-42b4-8e0b-b263dfd4cbde None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:30.796350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 462/922] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:30.800898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4cf1-0db0-4ab8-9512-3a5d5793868e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:30.801195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4cf1-0db0-4ab8-9512-3a5d5793868e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:30.801479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4cf1-0db0-4ab8-9512-3a5d5793868e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:30.819353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f31b4cf1-0db0-4ab8-9512-3a5d5793868e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:30.820683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f31b4cf1-0db0-4ab8-9512-3a5d5793868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:30.831621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:30.832629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:30.833158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:30.833639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:30.835169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:30.835693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:30.837315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:30.843300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 461/923] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:30.848962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:30.860459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:30.905593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 463/924] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:30.919914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c54e8b-e40d-4beb-8793-925f90a0ba72 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:30.920297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c54e8b-e40d-4beb-8793-925f90a0ba72 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:30.920581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c54e8b-e40d-4beb-8793-925f90a0ba72 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:30.925732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 464/925] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:30.927220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127121e-2816-435e-82a8-48c5325cac3c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:30.927599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127121e-2816-435e-82a8-48c5325cac3c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:30.927947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127121e-2816-435e-82a8-48c5325cac3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:30.946143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9127121e-2816-435e-82a8-48c5325cac3c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:30.947156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9127121e-2816-435e-82a8-48c5325cac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:30.954953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:30.972488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709bbc6-eaae-429b-aa9a-348822ac6b2e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.018258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.018927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.019213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.019490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.020232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:31.020551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.022262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.022850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 465/926] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:31.024021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127121e-2816-435e-82a8-48c5325cac3c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.032883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 462/927] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:31.039566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.118062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:31.148182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127121e-2816-435e-82a8-48c5325cac3c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.153327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e6d4a-5f4c-4253-957f-30ec558dca4a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.183525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257242b-c62c-45f8-b1ec-e9f2da4ee799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.184606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257242b-c62c-45f8-b1ec-e9f2da4ee799 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.185059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257242b-c62c-45f8-b1ec-e9f2da4ee799 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.185500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257242b-c62c-45f8-b1ec-e9f2da4ee799 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.191865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.192558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.192839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.193146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.194393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] RBAC: Adding query filter params (interface=public, service_id=eb9ef42dc557490e9017c668a86bb9af, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:31.194710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.196089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.197735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257242b-c62c-45f8-b1ec-e9f2da4ee799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:31.201432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 463/928] 10.222.0.32 () {56 vars in 1251 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/endpoints?service_id=eb9ef42dc557490e9017c668a86bb9af&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:31.202218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0257242b-c62c-45f8-b1ec-e9f2da4ee799 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:31.206637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:31.207543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:31.208454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:31.211464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 466/929] 10.222.0.32 () {56 vars in 1120 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:31.212454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257242b-c62c-45f8-b1ec-e9f2da4ee799 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.218718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:31.324855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf80aa-1830-452e-8f49-bfa4846fd47c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.331622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257242b-c62c-45f8-b1ec-e9f2da4ee799 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.361453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.362272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.362562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.362842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.363504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.364903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.367340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:31.372304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e11cc-3f6f-4e58-bf7b-4e721543f90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.373511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e11cc-3f6f-4e58-bf7b-4e721543f90f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.373972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e11cc-3f6f-4e58-bf7b-4e721543f90f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.374431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e11cc-3f6f-4e58-bf7b-4e721543f90f None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.375284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e11cc-3f6f-4e58-bf7b-4e721543f90f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.376836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e11cc-3f6f-4e58-bf7b-4e721543f90f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:31.378735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:31.379784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 30 11:51:31.381580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:31.389808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 464/930] 10.222.0.32 () {56 vars in 1132 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:31.389808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.396873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 467/931] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 11:51:31.400975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e11cc-3f6f-4e58-bf7b-4e721543f90f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.484333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:31.528561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e11cc-3f6f-4e58-bf7b-4e721543f90f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.542356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b6c23-8eff-4452-8850-1bc615716ed7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.575476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.576184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.576481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.576758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.577797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:31.578111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.580118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.585678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.586615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 468/932] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:31.586736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.587052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.587468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.588283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:31.588604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.591855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.594263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.605103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 465/933] 10.222.0.32 () {56 vars in 1115 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:31.611636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.695025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:31.726491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741dfa3-6eae-4613-a943-5a990caef206 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.744609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a5acc-757a-49eb-b642-a0dfdb4234a9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.756836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:31.763714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc9925-e841-443d-9989-cfda7f7da750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.764473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9925-e841-443d-9989-cfda7f7da750 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.764756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9925-e841-443d-9989-cfda7f7da750 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.765032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9925-e841-443d-9989-cfda7f7da750 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.765741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc9925-e841-443d-9989-cfda7f7da750 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:31.766085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc9925-e841-443d-9989-cfda7f7da750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.767718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc9925-e841-443d-9989-cfda7f7da750 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.773861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 469/934] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:31.775104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d6cb-9739-450c-88aa-4e46c195fdb4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.775397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d6cb-9739-450c-88aa-4e46c195fdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.775675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d6cb-9739-450c-88aa-4e46c195fdb4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.777360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 470/935] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:31.780894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d6cb-9739-450c-88aa-4e46c195fdb4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.785715 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f50829-7446-4aa2-b3d6-f04efc6b3fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.786520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f50829-7446-4aa2-b3d6-f04efc6b3fcb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.786828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f50829-7446-4aa2-b3d6-f04efc6b3fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.787837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f50829-7446-4aa2-b3d6-f04efc6b3fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.799024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f50829-7446-4aa2-b3d6-f04efc6b3fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.800532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f50829-7446-4aa2-b3d6-f04efc6b3fcb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-67f50829-7446-4aa2-b3d6-f04efc6b3fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:31.818536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:31.819596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:31.820979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:31.820979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:31.820979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:31.820979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:31.820979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 466/936] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:31.820979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e799e5-611b-4ce9-aced-337d069d6285 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.821311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e799e5-611b-4ce9-aced-337d069d6285 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.821588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e799e5-611b-4ce9-aced-337d069d6285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.839746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-68e799e5-611b-4ce9-aced-337d069d6285 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:31.841205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-68e799e5-611b-4ce9-aced-337d069d6285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:31.903236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d6cb-9739-450c-88aa-4e46c195fdb4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:31.916047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 467/937] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:31.917167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e799e5-611b-4ce9-aced-337d069d6285 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:31.953425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec9ce7-3ee6-4db0-9374-d93a5871db82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:31.954147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec9ce7-3ee6-4db0-9374-d93a5871db82 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:31.956027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec9ce7-3ee6-4db0-9374-d93a5871db82 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:31.956027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec9ce7-3ee6-4db0-9374-d93a5871db82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:31.956027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec9ce7-3ee6-4db0-9374-d93a5871db82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:31.957172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec9ce7-3ee6-4db0-9374-d93a5871db82 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:31.962890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:31.981824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 471/938] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:51:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:51:31.983052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec9ce7-3ee6-4db0-9374-d93a5871db82 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:32.053309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e799e5-611b-4ce9-aced-337d069d6285 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:32.054229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:32.106855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:32.107537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:32.107819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:32.108095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:32.108856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:32.109170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:32.110572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:32.132901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 468/939] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:32.137678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:32.150570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec9ce7-3ee6-4db0-9374-d93a5871db82 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:32.204806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:32.205742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:32.206176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:32.206612 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:32.207332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:32.208917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:32.210521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING keystone.server.flask.common [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 30 11:51:32.229919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 30 11:51:32.229919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 30 11:51:32.229919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [parameters: {'id': '98848e73dc484d4583b4ed626f1a0e1b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 30 11:51:32.229919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 98848e73dc484d4583b4ed626f1a0e1b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 98848e73dc484d4583b4ed626f1a0e1b. May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 30 11:51:32.238529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 30 11:51:32.239384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return project_ref.to_dict() May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() May 30 11:51:32.240994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 30 11:51:32.242570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application c = connection._execute_20( May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 30 11:51:32.243519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 30 11:51:32.244390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'id': '98848e73dc484d4583b4ed626f1a0e1b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:32.246391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:32.247410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict( May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 98848e73dc484d4583b4ed626f1a0e1b. May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 472/940] 10.222.0.32 () {60 vars in 1140 bytes} [Mon May 30 11:51:31 2022] POST /identity/v3/projects => generated 262 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:32.252839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:32.285346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:32.293142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718ee8f-ecbe-4c45-9120-845ff1fb5f66 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:32.342204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:32.343194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:32.343607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:32.343981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:32.352990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:32.354666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:32.371499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:32.373131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:32.374141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:32.374890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 469/941] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:51:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:32.380075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:32.386444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e683318-9953-4c23-9dc4-8c4ada51cf53 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:32.396536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:32.437655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:32.439011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:32.439481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:32.439972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:32.449753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:32.451956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:32.470735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:32.471673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:32.472597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:32.472597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:32.472597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:32.472597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:32.472597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:32.472597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:32.472597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:32.472597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:32.473882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 473/942] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:32.479644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:32.508730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6b84-533e-48cf-a5cc-53f2abf101a9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:32.561429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:32.562486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:32.562973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:32.563454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:32.564563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:32.565099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:32.567275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:32.587498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 470/943] 10.222.0.32 () {56 vars in 1149 bytes} [Mon May 30 11:51:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:32.592623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:32.606824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c4b65-752c-40fa-9490-924f0fb646cf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:32.615713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:32.650489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:32.651534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:32.652063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:32.652579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:32.653649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:32.654202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:32.655837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:32.681446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 474/944] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:32.718136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb812-3a92-4a0e-8207-bc380ea08d26 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:32.757933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:32.758679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:32.758964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:32.759242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:32.772290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:32.774056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:32.793995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:32.795129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:32.796006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:32.796006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:32.796006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:32.796006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:32.796006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 471/945] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:32.803120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:32.838052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:32.926362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0656b1f-0c73-4e20-a3d9-1215db626488 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:32.963378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:32.964055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:32.964373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:32.964650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:32.965369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:32.965678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:32.967044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:32.980784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 475/946] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:32.985989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:33.060951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:33.096416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0b6fc-a0b8-40f4-9201-3817e390c448 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:33.132773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:33.133590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:33.133873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:33.134148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:33.141506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:33.143218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:33.157659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:33.158585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:33.159485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 472/947] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:33.165446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:33.257334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:33.280982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d2671-bd5c-4a05-b2be-5aa01a7e8823 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:33.325669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7c8e3-b9cc-4878-9bc4-27ccdb0864a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:33.326331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7c8e3-b9cc-4878-9bc4-27ccdb0864a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:33.326621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7c8e3-b9cc-4878-9bc4-27ccdb0864a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:33.326906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7c8e3-b9cc-4878-9bc4-27ccdb0864a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:33.328034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7c8e3-b9cc-4878-9bc4-27ccdb0864a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:33.328606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7c8e3-b9cc-4878-9bc4-27ccdb0864a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:33.330067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7c8e3-b9cc-4878-9bc4-27ccdb0864a9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:33.343624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 476/948] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:33.349140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:33.460962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e785-3606-459e-9e22-84d28b42db99 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:33.490215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:33.498403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89cf97-c1b2-4e83-a6f6-7256a4f8f0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:33.499337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89cf97-c1b2-4e83-a6f6-7256a4f8f0ad None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:33.499625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89cf97-c1b2-4e83-a6f6-7256a4f8f0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:33.500373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89cf97-c1b2-4e83-a6f6-7256a4f8f0ad None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/d087549c76ae4c3896c7c8c6b5a74363/roles/8de96040db1e4e6e8c527b555d4c885a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:33.503071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89cf97-c1b2-4e83-a6f6-7256a4f8f0ad None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=d087549c76ae4c3896c7c8c6b5a74363, role_id=8de96040db1e4e6e8c527b555d4c885a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:33.504606 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89cf97-c1b2-4e83-a6f6-7256a4f8f0ad None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:33.523664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 473/949] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:51:33 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/d087549c76ae4c3896c7c8c6b5a74363/roles/8de96040db1e4e6e8c527b555d4c885a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:51:33.857151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6413509-b5ac-45b5-8c3e-4de1e3a9963b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:33.857654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6413509-b5ac-45b5-8c3e-4de1e3a9963b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:33.857873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6413509-b5ac-45b5-8c3e-4de1e3a9963b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:33.859618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 477/950] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:33.865105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d70c-4ea7-4e1b-8f98-e3ec7770eb73 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:33.865514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d70c-4ea7-4e1b-8f98-e3ec7770eb73 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:33.865896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4d70c-4ea7-4e1b-8f98-e3ec7770eb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:33.883779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-00e4d70c-4ea7-4e1b-8f98-e3ec7770eb73 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:33.885031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-00e4d70c-4ea7-4e1b-8f98-e3ec7770eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:33.957138 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 474/951] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:33.981197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6413509-b5ac-45b5-8c3e-4de1e3a9963b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:34.103094 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6413509-b5ac-45b5-8c3e-4de1e3a9963b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:34.140446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:34.141265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:34.141552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:34.142085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:34.142745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:34.144129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:34.157033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:34.159608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:34.161723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 478/952] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:34.168795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4d70c-4ea7-4e1b-8f98-e3ec7770eb73 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:34.269852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4d70c-4ea7-4e1b-8f98-e3ec7770eb73 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:34.289487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:34.308001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a7644-4919-4607-8991-94eab0d9be9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:34.308716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a7644-4919-4607-8991-94eab0d9be9d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:34.309002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a7644-4919-4607-8991-94eab0d9be9d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:34.309282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a7644-4919-4607-8991-94eab0d9be9d None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:34.310060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a7644-4919-4607-8991-94eab0d9be9d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:34.310374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a7644-4919-4607-8991-94eab0d9be9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:34.311833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a7644-4919-4607-8991-94eab0d9be9d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:34.317468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 475/953] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:34.323509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:34.337866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:34.435180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177d92f-c534-477b-9411-d6b978e190bc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:34.442575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b06a71-4cc4-4274-88cb-d1b7c7466f1c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:34.442859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b06a71-4cc4-4274-88cb-d1b7c7466f1c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:34.443817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b06a71-4cc4-4274-88cb-d1b7c7466f1c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:34.446974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 476/954] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:34.451317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cc8e-411e-4634-9b9f-86bee99ca28a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:34.451711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cc8e-411e-4634-9b9f-86bee99ca28a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:34.452111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67cc8e-411e-4634-9b9f-86bee99ca28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:34.471141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5a67cc8e-411e-4634-9b9f-86bee99ca28a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:34.473405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5a67cc8e-411e-4634-9b9f-86bee99ca28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:34.474335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:34.475343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:34.475680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:34.475962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:34.476864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:34.477125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:34.478883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:34.488188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 479/955] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:34.493124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:34.554785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 477/956] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:34.577485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67cc8e-411e-4634-9b9f-86bee99ca28a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:34.602216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:34.642468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0c001-a971-4cb6-ac31-b29127f5efd4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:34.695452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe94b7-4116-45e3-a63f-b3ec59362da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:34.697123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe94b7-4116-45e3-a63f-b3ec59362da8 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:34.697899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe94b7-4116-45e3-a63f-b3ec59362da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:34.698377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe94b7-4116-45e3-a63f-b3ec59362da8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:34.699631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe94b7-4116-45e3-a63f-b3ec59362da8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:34.701897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe94b7-4116-45e3-a63f-b3ec59362da8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:34.710360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67cc8e-411e-4634-9b9f-86bee99ca28a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:34.727227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:34.740348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 480/957] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:51:34 2022] POST /identity/v3/endpoints => generated 349 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:51:34.770352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:34.771259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:34.771539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:34.771889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:34.772649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:34.774268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:34.788496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:34.789681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:34.790902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 478/958] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:34.798486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe94b7-4116-45e3-a63f-b3ec59362da8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:34.931615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe94b7-4116-45e3-a63f-b3ec59362da8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:34.966877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:34.973690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e41414-457f-4dd5-a638-f90690f45fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:34.974729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e41414-457f-4dd5-a638-f90690f45fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:34.975389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e41414-457f-4dd5-a638-f90690f45fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:34.976064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e41414-457f-4dd5-a638-f90690f45fe7 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:34.977117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e41414-457f-4dd5-a638-f90690f45fe7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:34.977632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e41414-457f-4dd5-a638-f90690f45fe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:34.979258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e41414-457f-4dd5-a638-f90690f45fe7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:34.987457 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 481/959] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:34.992560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:35.105790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2108d-e0a1-4789-bfd2-d302146234c8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:35.141718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c536c-add7-4600-9430-9671484b0948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:35.143040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c536c-add7-4600-9430-9671484b0948 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:35.143547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c536c-add7-4600-9430-9671484b0948 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:35.144041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c536c-add7-4600-9430-9671484b0948 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:35.144984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c536c-add7-4600-9430-9671484b0948 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:35.145499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c536c-add7-4600-9430-9671484b0948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:35.147310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c536c-add7-4600-9430-9671484b0948 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:35.152936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 479/960] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:35.187093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774befc-02bb-4143-898d-c70ce26644ae None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:35.187422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774befc-02bb-4143-898d-c70ce26644ae None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:35.187667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774befc-02bb-4143-898d-c70ce26644ae None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:35.189532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 482/961] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:35.195935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f737d1-e67b-48fd-9845-d8be8e505487 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:35.196311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f737d1-e67b-48fd-9845-d8be8e505487 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:35.196583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f737d1-e67b-48fd-9845-d8be8e505487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:35.214950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67f737d1-e67b-48fd-9845-d8be8e505487 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:35.216212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67f737d1-e67b-48fd-9845-d8be8e505487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:35.252137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:35.298887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 480/962] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:35.324694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774befc-02bb-4143-898d-c70ce26644ae None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:35.459125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774befc-02bb-4143-898d-c70ce26644ae None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:35.500611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:35.501660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:35.502233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:35.502706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:35.503708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:35.505513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:35.521579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 30 11:51:35.521579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 30 11:51:35.521579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [parameters: {'user_id': '032e8c33fb3b48f4932eaf41d489ab96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 30 11:51:35.521579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 30 11:51:35.531010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 30 11:51:35.531972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 30 11:51:35.532951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 30 11:51:35.533870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = connection._execute_20( May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 30 11:51:35.534802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 30 11:51:35.537719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'user_id': '032e8c33fb3b48f4932eaf41d489ab96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:35.539840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 30 11:51:35.541773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 30 11:51:35.544065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 30 11:51:35.544065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 30 11:51:35.544065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 30 11:51:35.544065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 30 11:51:35.544065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 30 11:51:35.544065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:35.544065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 483/963] 10.222.0.32 () {60 vars in 1131 bytes} [Mon May 30 11:51:35 2022] POST /identity/v3/users => generated 160 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:35.548187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f737d1-e67b-48fd-9845-d8be8e505487 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:35.674239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f737d1-e67b-48fd-9845-d8be8e505487 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:35.716106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5dca7-1bed-4afc-a318-5731bf3091c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:35.717102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5dca7-1bed-4afc-a318-5731bf3091c3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:35.717393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5dca7-1bed-4afc-a318-5731bf3091c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:35.717677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5dca7-1bed-4afc-a318-5731bf3091c3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:35.727746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5dca7-1bed-4afc-a318-5731bf3091c3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:35.729304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5dca7-1bed-4afc-a318-5731bf3091c3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-21b5dca7-1bed-4afc-a318-5731bf3091c3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:35.746171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:35.747198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:35.748850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 481/964] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:35.754388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:35.821385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:35.876585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c50423-09ac-41fa-ac26-6e6d95e92aad None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:35.915310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df565985-bd43-42b0-aef9-1feaf4d6ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:35.916422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df565985-bd43-42b0-aef9-1feaf4d6ad88 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:35.916912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df565985-bd43-42b0-aef9-1feaf4d6ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:35.917415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df565985-bd43-42b0-aef9-1feaf4d6ad88 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:35.918434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df565985-bd43-42b0-aef9-1feaf4d6ad88 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:35.918989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df565985-bd43-42b0-aef9-1feaf4d6ad88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:35.920916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df565985-bd43-42b0-aef9-1feaf4d6ad88 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:35.962954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 484/965] 10.222.0.32 () {56 vars in 1101 bytes} [Mon May 30 11:51:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:35.970070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc550000-e00c-474a-aace-e31d3be6891d None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:35.970070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc550000-e00c-474a-aace-e31d3be6891d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:35.970070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc550000-e00c-474a-aace-e31d3be6891d None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:35.970070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 482/966] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:35.977008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3050-1e3c-4262-8c52-e87a9c10af43 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:35.977336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3050-1e3c-4262-8c52-e87a9c10af43 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:35.977577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3050-1e3c-4262-8c52-e87a9c10af43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:35.997066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bfbd3050-1e3c-4262-8c52-e87a9c10af43 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:35.998674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bfbd3050-1e3c-4262-8c52-e87a9c10af43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:36.085716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 485/967] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:36.113270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc550000-e00c-474a-aace-e31d3be6891d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:36.147587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:36.237535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc550000-e00c-474a-aace-e31d3be6891d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:36.278414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1597e-dde8-45f3-9d7a-14c714db89be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:36.279240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1597e-dde8-45f3-9d7a-14c714db89be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:36.279521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1597e-dde8-45f3-9d7a-14c714db89be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:36.279796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1597e-dde8-45f3-9d7a-14c714db89be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:36.292000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1597e-dde8-45f3-9d7a-14c714db89be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-bee1597e-dde8-45f3-9d7a-14c714db89be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:36.300467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:36.301405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:36.302268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:36.303964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 483/968] 10.222.0.32 () {56 vars in 1120 bytes} [Mon May 30 11:51:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:36.308998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd3050-1e3c-4262-8c52-e87a9c10af43 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:36.364252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:36.423364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd3050-1e3c-4262-8c52-e87a9c10af43 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:36.465396 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:36.466088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:36.466370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:36.466651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:36.467416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:36.467732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:36.469186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:36.483247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 486/969] 10.222.0.32 () {56 vars in 1115 bytes} [Mon May 30 11:51:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:36.484595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1597e-dde8-45f3-9d7a-14c714db89be None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:36.582809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:36.601790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1597e-dde8-45f3-9d7a-14c714db89be None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:36.639394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:36.640113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:36.640428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:36.640708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:36.653328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:36.654700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:36.673511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:36.674604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:36.675542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:36.675542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:36.675542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:36.675542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:36.676394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 484/970] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:36.680650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:36.801896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5609f3-53bd-4e5f-8c6e-bb213020aaf7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:36.809568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:36.839938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:36.840633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:36.840917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:36.841193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:36.841964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:36.842277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:36.843662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:36.862604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 487/971] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:36.864297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:36.993790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457a325-19dc-4949-847f-d13b2c30795e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:37.029793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:37.030604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.030882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.031155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.040026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:37.041533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:37.043334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:37.058313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:37.059453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 30 11:51:37.060442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:37.061626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 485/972] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:51:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:37.065309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:37.171037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6574196-23c7-49ba-a0f8-63dae8d67835 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:37.205228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c82024-1438-4f32-817a-faf884971838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:37.205913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82024-1438-4f32-817a-faf884971838 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.206202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82024-1438-4f32-817a-faf884971838 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.206485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82024-1438-4f32-817a-faf884971838 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.207333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c82024-1438-4f32-817a-faf884971838 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:37.207650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c82024-1438-4f32-817a-faf884971838 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:37.209835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c82024-1438-4f32-817a-faf884971838 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:37.228554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 488/973] 10.222.0.32 () {56 vars in 1149 bytes} [Mon May 30 11:51:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:37.234001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:37.286416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4238c9-b73c-4130-aaa6-cb98f50e5686 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.286768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4238c9-b73c-4130-aaa6-cb98f50e5686 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.287037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4238c9-b73c-4130-aaa6-cb98f50e5686 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.288678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 489/974] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:37.294468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923346f4-deb1-4026-a7be-88ed7f5e866d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.294761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923346f4-deb1-4026-a7be-88ed7f5e866d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.295047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923346f4-deb1-4026-a7be-88ed7f5e866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.313172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-923346f4-deb1-4026-a7be-88ed7f5e866d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:37.314416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-923346f4-deb1-4026-a7be-88ed7f5e866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:37.335705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:37.357042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9170ce-8690-4441-bf95-2aac1a89a288 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:37.389845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 490/975] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:37.392675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d22dde-f7d8-411d-9090-39f9917a67bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:37.393336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d22dde-f7d8-411d-9090-39f9917a67bf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.393623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d22dde-f7d8-411d-9090-39f9917a67bf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.393900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d22dde-f7d8-411d-9090-39f9917a67bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.413909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923346f4-deb1-4026-a7be-88ed7f5e866d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:37.465235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:37.468877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d22dde-f7d8-411d-9090-39f9917a67bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:37.470841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d22dde-f7d8-411d-9090-39f9917a67bf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:37.529669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923346f4-deb1-4026-a7be-88ed7f5e866d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:37.567383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:37.568744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.569250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.569539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.570200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:37.574285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:37.588392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:37.589470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:37.593303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 491/976] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:37.594613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 486/977] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:37 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:37.598155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:37.599526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d22dde-f7d8-411d-9090-39f9917a67bf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:37.620829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:37.706112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:37.710933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d22dde-f7d8-411d-9090-39f9917a67bf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:37.713811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cd9bd-d20a-4938-9ad7-555b1a0a81a6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:37.747132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b13e5-337a-443a-ad95-e6ba21a7e2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:37.747569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eafff1-0c9b-4853-8e11-07346f7f04f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:37.747932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b13e5-337a-443a-ad95-e6ba21a7e2fc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.748246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b13e5-337a-443a-ad95-e6ba21a7e2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.748350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eafff1-0c9b-4853-8e11-07346f7f04f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.748525 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b13e5-337a-443a-ad95-e6ba21a7e2fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.748630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eafff1-0c9b-4853-8e11-07346f7f04f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.748873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eafff1-0c9b-4853-8e11-07346f7f04f3 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.749581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eafff1-0c9b-4853-8e11-07346f7f04f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:37.749892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eafff1-0c9b-4853-8e11-07346f7f04f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:37.751282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eafff1-0c9b-4853-8e11-07346f7f04f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:37.756904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 492/978] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:37.758155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92409e7-68cd-49db-8882-a5c8dd5fbaa2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.758447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92409e7-68cd-49db-8882-a5c8dd5fbaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.758725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92409e7-68cd-49db-8882-a5c8dd5fbaa2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.760151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 493/979] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:37.762584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92409e7-68cd-49db-8882-a5c8dd5fbaa2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:37.764516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b13e5-337a-443a-ad95-e6ba21a7e2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:37.766009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b13e5-337a-443a-ad95-e6ba21a7e2fc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a80b13e5-337a-443a-ad95-e6ba21a7e2fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:37.784100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:37.785151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:37.786063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:37.786063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:37.786063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:37.786063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:37.786063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 487/980] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:37.786657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5886-51d6-4345-aa39-649706fdabe3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.786948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5886-51d6-4345-aa39-649706fdabe3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.787237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5886-51d6-4345-aa39-649706fdabe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.803933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-71ac5886-51d6-4345-aa39-649706fdabe3 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:37.804827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-71ac5886-51d6-4345-aa39-649706fdabe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:37.853032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:37.874055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92409e7-68cd-49db-8882-a5c8dd5fbaa2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:37.877262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 488/981] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:37.878122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac5886-51d6-4345-aa39-649706fdabe3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:37.914146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:37.914824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:37.915110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:37.915392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:37.916103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:37.916451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:37.917840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:37.922833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 494/982] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:37.923619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:37.962452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:37.983899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac5886-51d6-4345-aa39-649706fdabe3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:38.018813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:38.019474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:38.019758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:38.020041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:38.020789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:38.021205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:38.022583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:38.033403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92da1-bdaf-4eee-ba38-205153e88087 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:38.033954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 489/983] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:38.034871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:38.069606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728beaf3-862c-4965-aba7-cde4278a8fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:38.070165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728beaf3-862c-4965-aba7-cde4278a8fa4 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:38.070165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728beaf3-862c-4965-aba7-cde4278a8fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:38.070357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728beaf3-862c-4965-aba7-cde4278a8fa4 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:38.071640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728beaf3-862c-4965-aba7-cde4278a8fa4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:38.072514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728beaf3-862c-4965-aba7-cde4278a8fa4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:38.087797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 495/984] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:51:37 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:38.088909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728beaf3-862c-4965-aba7-cde4278a8fa4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:38.108291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:38.149763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc22e7-f637-448d-8078-dc9ca977451f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:38.199991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:38.200820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:38.201172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:38.201455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:38.202290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] RBAC: Adding query filter params (interface=admin, service_id=eb9ef42dc557490e9017c668a86bb9af, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:38.202596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:38.204078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:38.207846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:38.216029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 490/985] 10.222.0.32 () {56 vars in 1249 bytes} [Mon May 30 11:51:38 2022] GET /identity/v3/endpoints?service_id=eb9ef42dc557490e9017c668a86bb9af&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:38.217785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:38.230214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728beaf3-862c-4965-aba7-cde4278a8fa4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:38.271349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:38.273053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:38.273342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:38.273618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:38.281690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:38.282883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:38.300055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:38.301044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:38.301933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 496/986] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:38.307572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:38.339083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44cf4d-a2f0-4d19-bbb0-26f1e97ee078 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:38.377030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf94a71-a0dc-4fa5-babb-1ef1c96952c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:38.377563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf94a71-a0dc-4fa5-babb-1ef1c96952c8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:38.377824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf94a71-a0dc-4fa5-babb-1ef1c96952c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:38.378106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf94a71-a0dc-4fa5-babb-1ef1c96952c8 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:38.378796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf94a71-a0dc-4fa5-babb-1ef1c96952c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:38.380215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf94a71-a0dc-4fa5-babb-1ef1c96952c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:38.382018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:38.387770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 491/987] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:51:38 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 11:51:38.423797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f21fe7-e0ee-4846-8e43-a78bfb651688 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:38.465725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ff39b-880d-4168-b368-e9971e719498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:38.466433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff39b-880d-4168-b368-e9971e719498 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:38.466732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff39b-880d-4168-b368-e9971e719498 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:38.466986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ff39b-880d-4168-b368-e9971e719498 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:38.467797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff39b-880d-4168-b368-e9971e719498 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:38.468120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff39b-880d-4168-b368-e9971e719498 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:38.469624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ff39b-880d-4168-b368-e9971e719498 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:38.483490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 497/988] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:38.489377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf94a71-a0dc-4fa5-babb-1ef1c96952c8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:38.562183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:38.613894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf94a71-a0dc-4fa5-babb-1ef1c96952c8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:38.652431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4f3c-452a-445e-b81b-e11ebf896ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:38.653011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4f3c-452a-445e-b81b-e11ebf896ccc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:38.653292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4f3c-452a-445e-b81b-e11ebf896ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:38.653563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4f3c-452a-445e-b81b-e11ebf896ccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:38.654388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4f3c-452a-445e-b81b-e11ebf896ccc None admin] RBAC: Adding query filter params (role.id=8de96040db1e4e6e8c527b555d4c885a, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=d087549c76ae4c3896c7c8c6b5a74363) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:38.654698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4f3c-452a-445e-b81b-e11ebf896ccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:38.656134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd4f3c-452a-445e-b81b-e11ebf896ccc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:38.662983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 492/989] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:38 2022] GET /identity/v3/role_assignments?user.id=d087549c76ae4c3896c7c8c6b5a74363&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=8de96040db1e4e6e8c527b555d4c885a => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:40.408753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5ef2-da2c-469f-b4a7-632b1b03402b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:40.409289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5ef2-da2c-469f-b4a7-632b1b03402b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:40.409289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d5ef2-da2c-469f-b4a7-632b1b03402b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:40.412195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 498/990] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:40.415958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e37a5-ffaa-450a-9aa1-dcc6d7b3c6e9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:40.416428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e37a5-ffaa-450a-9aa1-dcc6d7b3c6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:40.416596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e37a5-ffaa-450a-9aa1-dcc6d7b3c6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:40.434567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cc9e37a5-ffaa-450a-9aa1-dcc6d7b3c6e9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:40.435823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cc9e37a5-ffaa-450a-9aa1-dcc6d7b3c6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:40.508434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 493/991] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:40.532011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d5ef2-da2c-469f-b4a7-632b1b03402b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:40.649625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d5ef2-da2c-469f-b4a7-632b1b03402b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:40.679010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:40.688992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:40.689801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:40.690092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:40.690373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] PATH_INFO: `/v3/services/compute` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:40.691035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:40.692452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:40.695002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191c72-7702-4e0e-bdb4-87d6cf52f07f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:40.695294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191c72-7702-4e0e-bdb4-87d6cf52f07f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:40.695574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191c72-7702-4e0e-bdb4-87d6cf52f07f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:40.696974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 494/992] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:40.701946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c4788-dd65-4d8f-b057-e420d4522a96 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:40.702231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c4788-dd65-4d8f-b057-e420d4522a96 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:40.702510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c4788-dd65-4d8f-b057-e420d4522a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:40.707608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:40.708725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:40.709582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 499/993] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:40.716356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:40.721323 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-750c4788-dd65-4d8f-b057-e420d4522a96 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:40.722511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-750c4788-dd65-4d8f-b057-e420d4522a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:40.784821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:40.800704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 495/994] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:40.824313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c4788-dd65-4d8f-b057-e420d4522a96 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:40.830271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d79915-8c6b-4f3d-8961-065cf0e1ecad None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:40.869526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:40.870190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:40.870480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:40.870762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:40.871475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:40.871792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:40.873221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:40.880249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 500/995] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:40.886875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:40.940785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c4788-dd65-4d8f-b057-e420d4522a96 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:40.946633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:40.979806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0914627-ad58-4c5a-8d1e-36a8794ac6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:40.980654 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914627-ad58-4c5a-8d1e-36a8794ac6cd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:40.980945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914627-ad58-4c5a-8d1e-36a8794ac6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:40.981225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914627-ad58-4c5a-8d1e-36a8794ac6cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:40.981891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0914627-ad58-4c5a-8d1e-36a8794ac6cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:40.983263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0914627-ad58-4c5a-8d1e-36a8794ac6cd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f0914627-ad58-4c5a-8d1e-36a8794ac6cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:40.995918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:40.997006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:40.997879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 496/996] 10.222.0.32 () {56 vars in 1126 bytes} [Mon May 30 11:51:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:40.998709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cad029-d606-4075-8688-d09df06860b1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:40.998791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a56b9-0f80-4490-b988-948c66e9b17a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:40.998791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a56b9-0f80-4490-b988-948c66e9b17a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:40.998933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a56b9-0f80-4490-b988-948c66e9b17a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.000154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 497/997] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:41.003815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a56b9-0f80-4490-b988-948c66e9b17a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:41.036182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8047e7b-8a04-4b5e-b37f-6beeba1178e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.037116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047e7b-8a04-4b5e-b37f-6beeba1178e1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.037471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047e7b-8a04-4b5e-b37f-6beeba1178e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.037750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8047e7b-8a04-4b5e-b37f-6beeba1178e1 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.038455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8047e7b-8a04-4b5e-b37f-6beeba1178e1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:41.038764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8047e7b-8a04-4b5e-b37f-6beeba1178e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.040139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8047e7b-8a04-4b5e-b37f-6beeba1178e1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.045148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 501/998] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:41.046390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a9e0e-21c9-4567-94b6-946d3953bd80 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.046684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a9e0e-21c9-4567-94b6-946d3953bd80 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.046968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a9e0e-21c9-4567-94b6-946d3953bd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.063762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-009a9e0e-21c9-4567-94b6-946d3953bd80 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:41.063977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:41.065040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-009a9e0e-21c9-4567-94b6-946d3953bd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:41.112865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a56b9-0f80-4490-b988-948c66e9b17a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:41.133345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 502/999] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:41.134198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a9e0e-21c9-4567-94b6-946d3953bd80 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:41.148881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.149787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.150081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.150360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.151068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:41.151380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.152781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.163320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 498/1000] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:41.163320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:41.188753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:41.239621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a9e0e-21c9-4567-94b6-946d3953bd80 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:41.259661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622e8b-c09f-443d-980f-37aa64b2e922 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:41.274650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.275326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.275613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.275902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.276109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:41.276726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] RBAC: Adding query filter params (interface=public, service_id=c3132b75d2174f35a2e39be38a417c27, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:41.277055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.278431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.283866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 503/1001] 10.222.0.32 () {56 vars in 1251 bytes} [Mon May 30 11:51:41 2022] GET /identity/v3/endpoints?service_id=c3132b75d2174f35a2e39be38a417c27&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:41.284815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:41.296253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.297058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.297341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.297616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] PATH_INFO: `/v3/services/image` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.298271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.299630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:41.311913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:41.312995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:41.313829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 499/1002] 10.222.0.32 () {56 vars in 1105 bytes} [Mon May 30 11:51:41 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:41.314853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:41.393117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f21106-9ffc-4d35-9573-58977d3d8ada None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:41.406967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:41.419204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452216fa-eba5-4fd9-af84-a79342ebf750 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:41.429809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.430751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.431088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.431395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.432109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:41.432451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.433830 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.438804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 504/1003] 10.222.0.32 () {56 vars in 1122 bytes} [Mon May 30 11:51:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:41.439601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:41.454574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fec57-366a-4e7c-b530-ba54c2b181ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.455246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fec57-366a-4e7c-b530-ba54c2b181ed None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.455530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fec57-366a-4e7c-b530-ba54c2b181ed None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.455812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fec57-366a-4e7c-b530-ba54c2b181ed None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.456538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fec57-366a-4e7c-b530-ba54c2b181ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.457900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fec57-366a-4e7c-b530-ba54c2b181ed None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.476193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 500/1004] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:51:41 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 11:51:41.476193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fec57-366a-4e7c-b530-ba54c2b181ed None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:41.534284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:41.554296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f863c-c389-4531-8b7b-24f842d8bdea None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:41.599915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.600637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.600925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.601205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.601961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] RBAC: Adding query filter params (name=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:41.602275 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.603688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.606217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fec57-366a-4e7c-b530-ba54c2b181ed None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:41.609235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 505/1005] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:41.615739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:41.655523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42413592-baec-4480-b78a-5c6bddac016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.656229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413592-baec-4480-b78a-5c6bddac016a None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.656525 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413592-baec-4480-b78a-5c6bddac016a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.656814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42413592-baec-4480-b78a-5c6bddac016a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.657563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42413592-baec-4480-b78a-5c6bddac016a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.659378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42413592-baec-4480-b78a-5c6bddac016a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.668865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:41.687552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 501/1006] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:51:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:41.747449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124901c-012b-427f-a43f-30a7a1f8a66c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:41.796088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc412a0-ea04-4239-aee4-dc8c9a110d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:41.796719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc412a0-ea04-4239-aee4-dc8c9a110d91 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:41.797004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc412a0-ea04-4239-aee4-dc8c9a110d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:41.797280 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc412a0-ea04-4239-aee4-dc8c9a110d91 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:41.798008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc412a0-ea04-4239-aee4-dc8c9a110d91 None admin] RBAC: Adding query filter params (type=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:41.798317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc412a0-ea04-4239-aee4-dc8c9a110d91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:41.799754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc412a0-ea04-4239-aee4-dc8c9a110d91 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:41.812189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 506/1007] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:43.787965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39288b-bdc3-4190-975c-0af1f506fda2 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:43.788559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39288b-bdc3-4190-975c-0af1f506fda2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:43.788649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39288b-bdc3-4190-975c-0af1f506fda2 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:43.789978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 502/1008] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:43.796024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645f611-05dd-4c7c-a92f-0d135621a262 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:43.796520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645f611-05dd-4c7c-a92f-0d135621a262 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:43.796693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645f611-05dd-4c7c-a92f-0d135621a262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:43.814336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e645f611-05dd-4c7c-a92f-0d135621a262 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:43.815538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e645f611-05dd-4c7c-a92f-0d135621a262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:43.888301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 507/1009] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:43 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:43.908930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39288b-bdc3-4190-975c-0af1f506fda2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:44.030037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ffffc-49f9-4349-9f52-a547fbdc6dbb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.030484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ffffc-49f9-4349-9f52-a547fbdc6dbb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.030944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ffffc-49f9-4349-9f52-a547fbdc6dbb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.036215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 508/1010] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:44.037265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39288b-bdc3-4190-975c-0af1f506fda2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:44.039008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a823-1361-4884-98a5-0c00b64deaa6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.039224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a823-1361-4884-98a5-0c00b64deaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.039539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a823-1361-4884-98a5-0c00b64deaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.042865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:44.058058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f265a823-1361-4884-98a5-0c00b64deaa6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:44.059262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f265a823-1361-4884-98a5-0c00b64deaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:44.080812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed99e-2ce1-4b9b-b3db-6db8c79e4fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:44.081616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed99e-2ce1-4b9b-b3db-6db8c79e4fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.081901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed99e-2ce1-4b9b-b3db-6db8c79e4fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.082176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed99e-2ce1-4b9b-b3db-6db8c79e4fc8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.082841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed99e-2ce1-4b9b-b3db-6db8c79e4fc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:44.084421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed99e-2ce1-4b9b-b3db-6db8c79e4fc8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:44.087222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-07fed99e-2ce1-4b9b-b3db-6db8c79e4fc8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:44.096519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:44.097577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:44.098416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 503/1011] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:44.105628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b87c-5630-425e-b92c-8a591c7130b8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.105918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b87c-5630-425e-b92c-8a591c7130b8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.107610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b87c-5630-425e-b92c-8a591c7130b8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.109158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 504/1012] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:44.111272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b87c-5630-425e-b92c-8a591c7130b8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:44.160059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 509/1013] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:44.160059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b9068-557f-48fe-bfa2-329e255a3ec6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.160059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b9068-557f-48fe-bfa2-329e255a3ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.160059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b9068-557f-48fe-bfa2-329e255a3ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.177017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-109b9068-557f-48fe-bfa2-329e255a3ec6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:44.178966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-109b9068-557f-48fe-bfa2-329e255a3ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:44.235122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b87c-5630-425e-b92c-8a591c7130b8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:44.262341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 510/1014] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:44.263287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b9068-557f-48fe-bfa2-329e255a3ec6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:44.277518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:44.278214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.278503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.278786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.279513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:44.279827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:44.284449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:44.290553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 505/1015] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:44.292238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:44.331390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:44.385904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:44.398993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b9068-557f-48fe-bfa2-329e255a3ec6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:44.409991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555bac5-819e-4b4b-a613-28e957dc3134 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:44.439111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2e752-b963-4c35-92b1-d48235c99481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:44.439832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2e752-b963-4c35-92b1-d48235c99481 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.440120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2e752-b963-4c35-92b1-d48235c99481 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.440437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2e752-b963-4c35-92b1-d48235c99481 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.441621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2e752-b963-4c35-92b1-d48235c99481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:44.443937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2e752-b963-4c35-92b1-d48235c99481 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:44.445907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 511/1016] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:44.446531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2e752-b963-4c35-92b1-d48235c99481 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:44.448042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10be1a-9742-4fdf-ba43-6284f0831798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:44.448742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10be1a-9742-4fdf-ba43-6284f0831798 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.449026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10be1a-9742-4fdf-ba43-6284f0831798 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.449301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10be1a-9742-4fdf-ba43-6284f0831798 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.450014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10be1a-9742-4fdf-ba43-6284f0831798 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:44.451365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10be1a-9742-4fdf-ba43-6284f0831798 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:44.466197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 506/1017] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:51:44 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:44.467334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10be1a-9742-4fdf-ba43-6284f0831798 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:44.568344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2e752-b963-4c35-92b1-d48235c99481 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:44.585894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:44.605146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10be1a-9742-4fdf-ba43-6284f0831798 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:44.612239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:44.612789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.613076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.613353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.614086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:44.614922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:44.616839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:44.623605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 512/1018] 10.222.0.32 () {56 vars in 1112 bytes} [Mon May 30 11:51:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:44.629152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:44.646711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386cd35-3c94-42b9-b806-62854c0e99f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:44.647698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386cd35-3c94-42b9-b806-62854c0e99f1 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.648040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386cd35-3c94-42b9-b806-62854c0e99f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.648354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386cd35-3c94-42b9-b806-62854c0e99f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.649097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386cd35-3c94-42b9-b806-62854c0e99f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:44.650577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386cd35-3c94-42b9-b806-62854c0e99f1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:44.657536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:44.682727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 507/1019] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:51:44 2022] POST /identity/v3/projects => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:51:44.753075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c59d3-8e8b-4510-a0c5-7f9075f22d14 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:44.796319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e65d5-c260-4762-87f1-782d19c907ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:44.797772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e65d5-c260-4762-87f1-782d19c907ea None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:44.798210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e65d5-c260-4762-87f1-782d19c907ea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:44.798639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e65d5-c260-4762-87f1-782d19c907ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:44.799566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e65d5-c260-4762-87f1-782d19c907ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:44.801242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e65d5-c260-4762-87f1-782d19c907ea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:44.823549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 513/1020] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:51:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:51:46.832215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3086d0-2b44-4f2a-bcbe-da54ecf0724c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:46.833718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3086d0-2b44-4f2a-bcbe-da54ecf0724c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:46.834014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3086d0-2b44-4f2a-bcbe-da54ecf0724c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:46.835406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 508/1021] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:46.840894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea411f9-25c7-4341-ace6-ae76b66e81a4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:46.841341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea411f9-25c7-4341-ace6-ae76b66e81a4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:46.841621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea411f9-25c7-4341-ace6-ae76b66e81a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:46.862802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5ea411f9-25c7-4341-ace6-ae76b66e81a4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:46.864051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5ea411f9-25c7-4341-ace6-ae76b66e81a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:46.953415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 514/1022] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:46.976532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3086d0-2b44-4f2a-bcbe-da54ecf0724c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.006019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30527032-70d6-4948-b0fb-9cd1be27d59c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.006312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30527032-70d6-4948-b0fb-9cd1be27d59c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.006600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30527032-70d6-4948-b0fb-9cd1be27d59c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.007987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 515/1023] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:47.013008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8371e7-3681-40f7-9f86-dcae36f077d1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.013297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8371e7-3681-40f7-9f86-dcae36f077d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.013580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8371e7-3681-40f7-9f86-dcae36f077d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.019644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:47.033047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ef8371e7-3681-40f7-9f86-dcae36f077d1 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:47.034885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ef8371e7-3681-40f7-9f86-dcae36f077d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:47.102852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3086d0-2b44-4f2a-bcbe-da54ecf0724c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.107924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 516/1024] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:47.133917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8371e7-3681-40f7-9f86-dcae36f077d1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.139152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.139945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.140253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.140517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] PATH_INFO: `/v3/services/image` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.141177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:47.143244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:47.156723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:47.157844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:47.158857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 509/1025] 10.222.0.32 () {56 vars in 1105 bytes} [Mon May 30 11:51:46 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:47.164207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.167768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:47.241976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8371e7-3681-40f7-9f86-dcae36f077d1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.270582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24f45c-cc9e-4221-9c90-640fdd8d5971 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.278391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71f2ec-0382-4bda-b298-289bb397c43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.279189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71f2ec-0382-4bda-b298-289bb397c43d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.279479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71f2ec-0382-4bda-b298-289bb397c43d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.279756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71f2ec-0382-4bda-b298-289bb397c43d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.280970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71f2ec-0382-4bda-b298-289bb397c43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:47.282439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71f2ec-0382-4bda-b298-289bb397c43d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:47.284061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 517/1026] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:47.285282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcb164-4028-4899-8251-16bdcb0b0150 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.285574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcb164-4028-4899-8251-16bdcb0b0150 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.285850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcb164-4028-4899-8251-16bdcb0b0150 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.288206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 518/1027] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:47.290683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcb164-4028-4899-8251-16bdcb0b0150 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.293921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:47.306768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dd1fa-9a9c-41a1-97ef-08273375df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.307445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dd1fa-9a9c-41a1-97ef-08273375df60 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.307735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dd1fa-9a9c-41a1-97ef-08273375df60 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.308015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dd1fa-9a9c-41a1-97ef-08273375df60 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.308764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dd1fa-9a9c-41a1-97ef-08273375df60 None admin] RBAC: Adding query filter params (name=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:47.309077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dd1fa-9a9c-41a1-97ef-08273375df60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:47.310453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dd1fa-9a9c-41a1-97ef-08273375df60 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:47.315306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 510/1028] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:47.316517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae834572-2b32-417f-82ba-f9fdfbdcd11d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.316808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae834572-2b32-417f-82ba-f9fdfbdcd11d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.317089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae834572-2b32-417f-82ba-f9fdfbdcd11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.334820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ae834572-2b32-417f-82ba-f9fdfbdcd11d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:47.336022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ae834572-2b32-417f-82ba-f9fdfbdcd11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:47.398017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:47.398920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcb164-4028-4899-8251-16bdcb0b0150 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.404039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 511/1029] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:47.404891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae834572-2b32-417f-82ba-f9fdfbdcd11d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.435230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98857c90-f3c7-4899-b7f6-9ff32e34c623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.435907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98857c90-f3c7-4899-b7f6-9ff32e34c623 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.436226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98857c90-f3c7-4899-b7f6-9ff32e34c623 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.436513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98857c90-f3c7-4899-b7f6-9ff32e34c623 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.437222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98857c90-f3c7-4899-b7f6-9ff32e34c623 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:47.438810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98857c90-f3c7-4899-b7f6-9ff32e34c623 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:47.459241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 519/1030] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:51:47 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:51:47.460542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98857c90-f3c7-4899-b7f6-9ff32e34c623 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.526933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae834572-2b32-417f-82ba-f9fdfbdcd11d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.540662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:47.569175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.569877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.570166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.570449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.571676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:47.571998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:47.573808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:47.580639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 512/1031] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:47.587372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.597624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98857c90-f3c7-4899-b7f6-9ff32e34c623 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.637659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5292dc4-4f23-472a-801a-87230da55757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.638489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292dc4-4f23-472a-801a-87230da55757 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.638775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292dc4-4f23-472a-801a-87230da55757 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.639145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292dc4-4f23-472a-801a-87230da55757 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.652540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5292dc4-4f23-472a-801a-87230da55757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d5292dc4-4f23-472a-801a-87230da55757 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:47.660698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:47.661988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:47.662851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 520/1032] 10.222.0.32 () {56 vars in 1120 bytes} [Mon May 30 11:51:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:47.668754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5292dc4-4f23-472a-801a-87230da55757 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.669229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:47.719433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e77590-3552-4564-b854-a51a684636f7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.759994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.760705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.760994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.761278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.762075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] RBAC: Adding query filter params (interface=public, service_id=cd3c0a42ad734501ac81b42847ce8fd0, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:47.762440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:47.763760 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:47.770190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 513/1033] 10.222.0.32 () {56 vars in 1251 bytes} [Mon May 30 11:51:47 2022] GET /identity/v3/endpoints?service_id=cd3c0a42ad734501ac81b42847ce8fd0&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:47.776512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.783478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5292dc4-4f23-472a-801a-87230da55757 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.804142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:47.827777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.829149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.829886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.830422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.831784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:47.832328 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:47.835406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:47.846704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 521/1034] 10.222.0.32 () {56 vars in 1115 bytes} [Mon May 30 11:51:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:47.852926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:47.893106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72016a0-825e-4b7c-9d1f-24c98c9c4fe3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:47.923380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:47.935832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565350b7-2781-4a21-957d-5ef3ae21f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:47.936545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565350b7-2781-4a21-957d-5ef3ae21f740 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:47.936833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565350b7-2781-4a21-957d-5ef3ae21f740 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:47.937109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565350b7-2781-4a21-957d-5ef3ae21f740 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:47.937806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565350b7-2781-4a21-957d-5ef3ae21f740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:47.939177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565350b7-2781-4a21-957d-5ef3ae21f740 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:47.946474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 514/1035] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:51:47 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 11:51:47.975807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a759-e528-4e0c-8ee5-5922dd2b19db None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:48.014384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:48.015217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:48.015507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:48.015791 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:48.028718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:48.030267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:48.050768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:48.051858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:48.053770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:48.053770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:48.053770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:48.053770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:48.053770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 522/1036] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:48.061006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565350b7-2781-4a21-957d-5ef3ae21f740 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:48.176254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565350b7-2781-4a21-957d-5ef3ae21f740 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:48.211239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:48.212291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:48.212690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:48.213184 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:48.214497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:48.215015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:48.216823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:48.229830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 515/1037] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:48.235668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:48.271261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:48.345395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888a7a9-7b11-4cc6-b9d2-a512901ee6ea None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:48.381354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:48.382159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:48.382452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:48.382806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:48.390646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:48.392387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:48.407044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:48.408027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:48.409061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:48.412238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 523/1038] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:51:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:48.414746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:48.467881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:48.521121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac2626-5f81-4a80-8e97-f92da81d33db None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:48.556045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:48.556752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:48.557040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:48.557317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:48.558158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:48.558473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:48.559907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:48.582250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 516/1039] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:51:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:48.584395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:48.683712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:48.693904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10a205-ad7d-4f0b-876b-1818f234dec1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:48.728984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a6bb4-e208-4d87-8985-ffe3bae73a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:48.729562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a6bb4-e208-4d87-8985-ffe3bae73a49 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:48.729850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a6bb4-e208-4d87-8985-ffe3bae73a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:48.730133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a6bb4-e208-4d87-8985-ffe3bae73a49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:48.799571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a6bb4-e208-4d87-8985-ffe3bae73a49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:48.801103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a6bb4-e208-4d87-8985-ffe3bae73a49 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:48.893548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:48.909120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 524/1040] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:48 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:48.910880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:49.015012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1f94a-2051-4a5f-ba2c-42b4c72d7dda None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:49.025650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:49.053470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:49.054269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:49.054553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:49.054830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:49.065253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:49.066697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:49.085728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:49.086750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:49.087680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:49.087680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:49.087680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:49.087680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:49.091856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 517/1041] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:49.093205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a6bb4-e208-4d87-8985-ffe3bae73a49 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:49.199360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a6bb4-e208-4d87-8985-ffe3bae73a49 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:49.235839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:49.236531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:49.236815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:49.237098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:49.237823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:49.238234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:49.240090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:49.257489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 525/1042] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:49.259072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:49.365830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca913cac-5247-4ecc-8645-84dc0d49f5a9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:49.405182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:49.405983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:49.406267 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:49.406544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:49.414509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:49.415903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:49.430914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:49.431901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:49.433200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:49.433200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:49.433200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:49.433200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:49.433200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:49.433200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:49.433200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:49.433200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:49.436483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 518/1043] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:49.437957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:49.496925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:49.549773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c51fe2-113e-4d5c-be8c-07a653bdf349 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:49.593072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a36541-3dfa-430c-8e21-8651beeccd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:49.593744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36541-3dfa-430c-8e21-8651beeccd09 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:49.594027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36541-3dfa-430c-8e21-8651beeccd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:49.594300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a36541-3dfa-430c-8e21-8651beeccd09 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:49.595085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a36541-3dfa-430c-8e21-8651beeccd09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:49.595394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a36541-3dfa-430c-8e21-8651beeccd09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:49.596915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a36541-3dfa-430c-8e21-8651beeccd09 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:49.616276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 526/1044] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:49.617600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:49.648956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:49.733332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc5591-90c6-4e15-ac34-8cb9f1d46ed6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:49.781094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe6f17-ef2a-48c3-b28d-2ac897bbfdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:49.781859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe6f17-ef2a-48c3-b28d-2ac897bbfdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:49.782160 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe6f17-ef2a-48c3-b28d-2ac897bbfdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:49.782443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe6f17-ef2a-48c3-b28d-2ac897bbfdc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:49.783290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe6f17-ef2a-48c3-b28d-2ac897bbfdc8 None admin] RBAC: Adding query filter params (role.id=8de96040db1e4e6e8c527b555d4c885a, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=70dce2f0284847ab8f07f55d700d0f60) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:49.783617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe6f17-ef2a-48c3-b28d-2ac897bbfdc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:49.785093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe6f17-ef2a-48c3-b28d-2ac897bbfdc8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:49.796408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 519/1045] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:49 2022] GET /identity/v3/role_assignments?user.id=70dce2f0284847ab8f07f55d700d0f60&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=8de96040db1e4e6e8c527b555d4c885a => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:49.832859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8b502-86f7-4796-be52-561475a98bdc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:49.834181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8b502-86f7-4796-be52-561475a98bdc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:49.834181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8b502-86f7-4796-be52-561475a98bdc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:49.835549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 527/1046] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:49.841674 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe92de0-4959-4b35-bd0c-b3612b646660 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:49.842021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe92de0-4959-4b35-bd0c-b3612b646660 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:49.842245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe92de0-4959-4b35-bd0c-b3612b646660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:49.862294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-efe92de0-4959-4b35-bd0c-b3612b646660 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:49.863524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-efe92de0-4959-4b35-bd0c-b3612b646660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:49.949806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 520/1047] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:49.975259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8b502-86f7-4796-be52-561475a98bdc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.094759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8b502-86f7-4796-be52-561475a98bdc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:50.109904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:50.135357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac24ee-fc60-4ae0-9af0-973dd5ecd5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:50.136553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac24ee-fc60-4ae0-9af0-973dd5ecd5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.136553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac24ee-fc60-4ae0-9af0-973dd5ecd5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.136865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac24ee-fc60-4ae0-9af0-973dd5ecd5b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.148284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac24ee-fc60-4ae0-9af0-973dd5ecd5b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3fac24ee-fc60-4ae0-9af0-973dd5ecd5b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:50.156070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:50.157115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:50.157958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:50.159392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 528/1048] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:50.163896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe92de0-4959-4b35-bd0c-b3612b646660 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.180108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:50.260728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218f06-d5cd-4355-8c85-ad68c353438a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.260938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218f06-d5cd-4355-8c85-ad68c353438a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.261223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12218f06-d5cd-4355-8c85-ad68c353438a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.262756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 529/1049] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:50.267788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e02310-fa41-4638-a446-c8dd47235864 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.268086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e02310-fa41-4638-a446-c8dd47235864 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.268409 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e02310-fa41-4638-a446-c8dd47235864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.272281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe92de0-4959-4b35-bd0c-b3612b646660 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:50.285948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-61e02310-fa41-4638-a446-c8dd47235864 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:50.287962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-61e02310-fa41-4638-a446-c8dd47235864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:50.317250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:50.318992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.319857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.320348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.321263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:50.321832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:50.323461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:50.332950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 521/1050] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:50.339254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.376322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 530/1051] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:50.390709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e02310-fa41-4638-a446-c8dd47235864 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.423768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:50.450725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659407f-e19d-4f6f-982e-9c98d7054ea0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:50.488253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dc276-b10a-4528-9a0b-877cc50446fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:50.489163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc276-b10a-4528-9a0b-877cc50446fb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.489967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc276-b10a-4528-9a0b-877cc50446fb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.490241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc276-b10a-4528-9a0b-877cc50446fb None admin] PATH_INFO: `/v3/users/38a5a25ab50c47fbaeff865cfbb078c0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.504619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dc276-b10a-4528-9a0b-877cc50446fb None admin] RBAC: Authorizing `identity:get_user(user_id=38a5a25ab50c47fbaeff865cfbb078c0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:50.506104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dc276-b10a-4528-9a0b-877cc50446fb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:50.507963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 522/1052] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:51:50 2022] GET /identity/v3/users/38a5a25ab50c47fbaeff865cfbb078c0 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:50.509688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e02310-fa41-4638-a446-c8dd47235864 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:50.511323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:50.513654 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dc276-b10a-4528-9a0b-877cc50446fb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.550595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:50.551393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.552246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.552739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] PATH_INFO: `/v3/services/image` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.553513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:50.554885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:50.566739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:50.568043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:50.569106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 531/1053] 10.222.0.32 () {56 vars in 1105 bytes} [Mon May 30 11:51:50 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:50.575115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.635762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dc276-b10a-4528-9a0b-877cc50446fb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:50.673536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab145c05-c101-4df6-9599-9fe053a81743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:50.674360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145c05-c101-4df6-9599-9fe053a81743 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.674835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145c05-c101-4df6-9599-9fe053a81743 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.675211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab145c05-c101-4df6-9599-9fe053a81743 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.678921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:50.685253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db229112-c10b-4d7e-9ea6-22397bbc874a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:50.719081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:50.719848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.720242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.720636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.721354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] RBAC: Adding query filter params (name=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:50.721668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:50.723079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:50.732818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 532/1054] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:50.734019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.757814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab145c05-c101-4df6-9599-9fe053a81743 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:50.759288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab145c05-c101-4df6-9599-9fe053a81743 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:50.765968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:50.838692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5ded-ea09-46ff-9d44-054f5d8d059b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:50.871893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 523/1055] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:50 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:50.872953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab145c05-c101-4df6-9599-9fe053a81743 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.882149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:50.882149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:50.882149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:50.882149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:50.884428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] RBAC: Adding query filter params (type=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:50.884428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:50.884428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:50.894636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 533/1056] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:50.895632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:50.913276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:50.985308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab145c05-c101-4df6-9599-9fe053a81743 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:51.000258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:51.010812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98222f9-2dde-42aa-b4eb-2f5967b34503 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:51.022222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f32914-e9d8-4c8f-92b2-0ce16f6ee918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:51.023468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f32914-e9d8-4c8f-92b2-0ce16f6ee918 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:51.024110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f32914-e9d8-4c8f-92b2-0ce16f6ee918 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:51.024512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f32914-e9d8-4c8f-92b2-0ce16f6ee918 None admin] PATH_INFO: `/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:51.025728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f32914-e9d8-4c8f-92b2-0ce16f6ee918 None admin] RBAC: Authorizing `identity:get_project(project_id=7f8a844b23f1431bbfe6b04e06dcf944)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:51.027298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f32914-e9d8-4c8f-92b2-0ce16f6ee918 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:51.032342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 524/1057] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:51:50 2022] GET /identity/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:51.034962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f32914-e9d8-4c8f-92b2-0ce16f6ee918 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:51.052253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35f208-fe8b-4aac-bf8a-cb3cde99ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:51.053118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f208-fe8b-4aac-bf8a-cb3cde99ee98 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:51.053362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f208-fe8b-4aac-bf8a-cb3cde99ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:51.053590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35f208-fe8b-4aac-bf8a-cb3cde99ee98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:51.054345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35f208-fe8b-4aac-bf8a-cb3cde99ee98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:51.055804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35f208-fe8b-4aac-bf8a-cb3cde99ee98 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:51.078517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 534/1058] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:51:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:51:51.160189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f32914-e9d8-4c8f-92b2-0ce16f6ee918 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:51.161985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:51.205824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952b1a1-5434-408d-a996-dc9d14d14a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:51.206495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b1a1-5434-408d-a996-dc9d14d14a54 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:51.206779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b1a1-5434-408d-a996-dc9d14d14a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:51.207058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952b1a1-5434-408d-a996-dc9d14d14a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:51.208567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952b1a1-5434-408d-a996-dc9d14d14a54 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=7f8a844b23f1431bbfe6b04e06dcf944, user.id=38a5a25ab50c47fbaeff865cfbb078c0) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:51.209088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952b1a1-5434-408d-a996-dc9d14d14a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:51.210586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952b1a1-5434-408d-a996-dc9d14d14a54 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:51.217059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 525/1059] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:51 2022] GET /identity/v3/role_assignments?user.id=38a5a25ab50c47fbaeff865cfbb078c0&scope.project.id=7f8a844b23f1431bbfe6b04e06dcf944&role.id=4d21498572b14478a996ac538be5382e => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:52.102155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f2ea-32f8-4613-b32d-411a393ccf13 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:52.102680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f2ea-32f8-4613-b32d-411a393ccf13 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:52.102761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f2ea-32f8-4613-b32d-411a393ccf13 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:52.104305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 535/1060] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:52.109388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dd5b0-3d03-47b7-9e86-ded291c26587 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:52.109834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dd5b0-3d03-47b7-9e86-ded291c26587 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:52.109912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dd5b0-3d03-47b7-9e86-ded291c26587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:52.127192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b25dd5b0-3d03-47b7-9e86-ded291c26587 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:52.128399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b25dd5b0-3d03-47b7-9e86-ded291c26587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:52.196048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 526/1061] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:52.218890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674f2ea-32f8-4613-b32d-411a393ccf13 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:52.330721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674f2ea-32f8-4613-b32d-411a393ccf13 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:52.372224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59360e-c94d-487c-9a33-c4d956f2b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:52.372965 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59360e-c94d-487c-9a33-c4d956f2b0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:52.373254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59360e-c94d-487c-9a33-c4d956f2b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:52.373537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59360e-c94d-487c-9a33-c4d956f2b0b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:52.385427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59360e-c94d-487c-9a33-c4d956f2b0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9e59360e-c94d-487c-9a33-c4d956f2b0b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:52.394029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:52.394968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:52.396862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 536/1062] 10.222.0.32 () {56 vars in 1120 bytes} [Mon May 30 11:51:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:52.402121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dd5b0-3d03-47b7-9e86-ded291c26587 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:52.498792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:52.511778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dd5b0-3d03-47b7-9e86-ded291c26587 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:52.547486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:52.548207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:52.548478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:52.548758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:52.549507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:52.549821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:52.551188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:52.564529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 527/1063] 10.222.0.32 () {56 vars in 1115 bytes} [Mon May 30 11:51:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:52.566087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59360e-c94d-487c-9a33-c4d956f2b0b8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:52.579599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:52.675317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59360e-c94d-487c-9a33-c4d956f2b0b8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:52.709674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:52.710483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:52.710774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:52.711059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:52.721868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:52.723343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:52.742367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:52.743326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:52.744305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:52.744305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:52.744305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:52.744305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:52.744305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 537/1064] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:52.749521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:52.857536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1be7d-2e30-4091-a448-4896d62ac095 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:52.894886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:52.895589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:52.896056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:52.896605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:52.897352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:52.897663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:52.899026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:52.916019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 528/1065] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:52.917578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:52.983514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:53.027223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b5a17-eac7-4b98-a8c2-230ca18ce6c1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:53.062671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:53.063523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.063819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.064103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.073463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:53.074960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:53.090055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:53.091105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:53.092075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 538/1066] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:51:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:53.098333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:53.117519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:53.217264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62152172-c163-4452-b0ef-397d03ef32a4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:53.256469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db1048-d8e4-40db-a4cd-b7a03bc012fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:53.257076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db1048-d8e4-40db-a4cd-b7a03bc012fd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.257365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db1048-d8e4-40db-a4cd-b7a03bc012fd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.257641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db1048-d8e4-40db-a4cd-b7a03bc012fd None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.258480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db1048-d8e4-40db-a4cd-b7a03bc012fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:53.258789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db1048-d8e4-40db-a4cd-b7a03bc012fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:53.260216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db1048-d8e4-40db-a4cd-b7a03bc012fd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:53.283259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 529/1067] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:51:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:53.284539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:53.394495 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e1975-7d82-48ca-8968-8a2a654f18c3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:53.418625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:53.431572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:53.432413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.432704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.432984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.444924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:53.445910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ce52e-afec-464a-8431-bcede639baaf None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.446208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ce52e-afec-464a-8431-bcede639baaf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.446410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:53.446509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ce52e-afec-464a-8431-bcede639baaf None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.447990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 530/1068] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:53.453260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a006c-5e3c-481d-a135-c73bf6c06834 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.453550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a006c-5e3c-481d-a135-c73bf6c06834 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.453828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a006c-5e3c-481d-a135-c73bf6c06834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:53.466137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:53.467139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:53.468058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:53.468058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:53.468058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:53.468058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:53.468058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 539/1069] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:53.472999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:53.473486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f14a006c-5e3c-481d-a135-c73bf6c06834 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:53.474689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f14a006c-5e3c-481d-a135-c73bf6c06834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:53.552762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 531/1070] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:53.554442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dcb30-5b92-42eb-bf7e-59c522e39c4f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.554736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dcb30-5b92-42eb-bf7e-59c522e39c4f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.555017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dcb30-5b92-42eb-bf7e-59c522e39c4f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.556456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 532/1071] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:53.561757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c38d3-057e-48e5-96ff-e3a570a2ef0c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.562383 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c38d3-057e-48e5-96ff-e3a570a2ef0c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.562383 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c38d3-057e-48e5-96ff-e3a570a2ef0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.568077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:53.581609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2c4c38d3-057e-48e5-96ff-e3a570a2ef0c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:53.583017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2c4c38d3-057e-48e5-96ff-e3a570a2ef0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:53.592108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aaf19-e675-491c-9f57-91d1a43c3292 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:53.628352 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:53.629025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.629308 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.629607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.630873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:53.631181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:53.632583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:53.645519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 540/1072] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:53.646361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:53.653279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 533/1073] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:53.654385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c38d3-057e-48e5-96ff-e3a570a2ef0c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:53.681457 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:53.747620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27a829-d1cf-4d8b-8fb5-dc34939780ed None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:53.758726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c38d3-057e-48e5-96ff-e3a570a2ef0c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:53.782371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dd83a-9c2c-4af2-a9fe-d5c83a2f585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:53.783025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dd83a-9c2c-4af2-a9fe-d5c83a2f585a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.783305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dd83a-9c2c-4af2-a9fe-d5c83a2f585a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.783579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dd83a-9c2c-4af2-a9fe-d5c83a2f585a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.788395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:53.793856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:53.794661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.794952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.795226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.802606 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:53.804108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:53.818454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:53.819384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 534/1074] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:53.820713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:53.856088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dd83a-9c2c-4af2-a9fe-d5c83a2f585a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:53.857584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0dd83a-9c2c-4af2-a9fe-d5c83a2f585a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:53.896442 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:53.928804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc3fb7-5a53-4435-b3bf-80850948c1b2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:53.962287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 541/1075] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:51:53.962897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b9c6f-76da-496f-a033-da34e0c8a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:53.963130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dd83a-9c2c-4af2-a9fe-d5c83a2f585a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:53.963699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b9c6f-76da-496f-a033-da34e0c8a770 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:53.963984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b9c6f-76da-496f-a033-da34e0c8a770 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:53.964295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b9c6f-76da-496f-a033-da34e0c8a770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:53.975518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b9c6f-76da-496f-a033-da34e0c8a770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ed2b9c6f-76da-496f-a033-da34e0c8a770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:53.983183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:53.984370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 535/1076] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:53.985236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b9c6f-76da-496f-a033-da34e0c8a770 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:54.042141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:54.071926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0dd83a-9c2c-4af2-a9fe-d5c83a2f585a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:54.092785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b9c6f-76da-496f-a033-da34e0c8a770 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:54.106617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:54.107288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:54.107577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:54.107856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:54.108693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:54.109009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:54.110465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:54.112087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:54.123653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 542/1077] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:54.124530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:54.135079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:54.135874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:54.136258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:54.136459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:54.148278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:54.149698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:54.167505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:54.168595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:54.169503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:54.169503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:54.169503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:54.169503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:54.169503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 536/1078] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:54.169503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:54.226996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed376-5a49-47dc-ad5d-726a52fcf9dd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:54.261631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:54.262299 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:54.262588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:54.262870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:54.263625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:54.263937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:54.265356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:54.274749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 543/1079] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:54.275588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:54.278152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f94cbd-502b-4cb2-aaa6-202c773e2892 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:54.280943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:54.314275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329c3f5-aa09-410b-93aa-da60cb349938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:54.315158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329c3f5-aa09-410b-93aa-da60cb349938 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:54.315438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329c3f5-aa09-410b-93aa-da60cb349938 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:54.315711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329c3f5-aa09-410b-93aa-da60cb349938 None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/70dce2f0284847ab8f07f55d700d0f60/roles/8de96040db1e4e6e8c527b555d4c885a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:54.318282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329c3f5-aa09-410b-93aa-da60cb349938 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=70dce2f0284847ab8f07f55d700d0f60, role_id=8de96040db1e4e6e8c527b555d4c885a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:54.319707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329c3f5-aa09-410b-93aa-da60cb349938 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:54.332408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 537/1080] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:51:54 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/70dce2f0284847ab8f07f55d700d0f60/roles/8de96040db1e4e6e8c527b555d4c885a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:51:54.332555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:54.337078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329c3f5-aa09-410b-93aa-da60cb349938 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:54.392407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1567a-cacc-4758-b074-dfc954ff23cf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:54.434430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:54.435115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:54.435407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:54.435689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:54.436486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:54.436803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:54.438240 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:54.453204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 544/1081] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:54.459264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329c3f5-aa09-410b-93aa-da60cb349938 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:54.501221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63228674-91c7-42e7-b2d2-b99ac3032ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:54.502133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63228674-91c7-42e7-b2d2-b99ac3032ceb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:54.502418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63228674-91c7-42e7-b2d2-b99ac3032ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:54.502696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63228674-91c7-42e7-b2d2-b99ac3032ceb None admin] PATH_INFO: `/v3/users/38a5a25ab50c47fbaeff865cfbb078c0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:54.504043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63228674-91c7-42e7-b2d2-b99ac3032ceb None admin] RBAC: Authorizing `identity:get_user(user_id=38a5a25ab50c47fbaeff865cfbb078c0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:54.505637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63228674-91c7-42e7-b2d2-b99ac3032ceb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:54.507598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 538/1082] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:51:54 2022] GET /identity/v3/users/38a5a25ab50c47fbaeff865cfbb078c0 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:54.513502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:54.641245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e3946-326f-4533-b460-b54f8b188c4b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:54.660521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:54.682734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8118b9-abfd-48ee-8035-66473280b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:54.683618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8118b9-abfd-48ee-8035-66473280b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:54.683909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8118b9-abfd-48ee-8035-66473280b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:54.684273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8118b9-abfd-48ee-8035-66473280b6a4 None admin] PATH_INFO: `/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:54.685476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8118b9-abfd-48ee-8035-66473280b6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=7f8a844b23f1431bbfe6b04e06dcf944)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:54.686939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8118b9-abfd-48ee-8035-66473280b6a4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:54.688722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 545/1083] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:51:54 2022] GET /identity/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:54.695341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63228674-91c7-42e7-b2d2-b99ac3032ceb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:54.729623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:54.807680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63228674-91c7-42e7-b2d2-b99ac3032ceb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:54.842578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7856d4-5529-4f18-bbeb-9268ca8b2af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:54.843392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7856d4-5529-4f18-bbeb-9268ca8b2af0 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:54.843676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7856d4-5529-4f18-bbeb-9268ca8b2af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:54.843956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7856d4-5529-4f18-bbeb-9268ca8b2af0 None admin] PATH_INFO: `/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944/users/38a5a25ab50c47fbaeff865cfbb078c0/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:54.846021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7856d4-5529-4f18-bbeb-9268ca8b2af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f8a844b23f1431bbfe6b04e06dcf944, user_id=38a5a25ab50c47fbaeff865cfbb078c0, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:54.847461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7856d4-5529-4f18-bbeb-9268ca8b2af0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:54.860872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 539/1084] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:51:54 2022] PUT /identity/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944/users/38a5a25ab50c47fbaeff865cfbb078c0/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:51:56.624598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aadf80-d6ef-4324-9824-e7c419a542a9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:56.625155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aadf80-d6ef-4324-9824-e7c419a542a9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:56.625155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aadf80-d6ef-4324-9824-e7c419a542a9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:56.626549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 546/1085] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:51:56.632795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c65d-65d2-4fe8-97a2-1589b7abc027 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:56.633271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c65d-65d2-4fe8-97a2-1589b7abc027 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:56.633376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c65d-65d2-4fe8-97a2-1589b7abc027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:56.650924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9811c65d-65d2-4fe8-97a2-1589b7abc027 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:56.652103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9811c65d-65d2-4fe8-97a2-1589b7abc027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:56.718588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 540/1086] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:51:56.743235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aadf80-d6ef-4324-9824-e7c419a542a9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:56.850358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aadf80-d6ef-4324-9824-e7c419a542a9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:56.888948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49616e84-95b8-4f03-895a-20760ccf7331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:56.889783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49616e84-95b8-4f03-895a-20760ccf7331 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:56.890069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49616e84-95b8-4f03-895a-20760ccf7331 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:56.890343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49616e84-95b8-4f03-895a-20760ccf7331 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:56.902093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49616e84-95b8-4f03-895a-20760ccf7331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-49616e84-95b8-4f03-895a-20760ccf7331 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:56.910208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:56.911478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:56.912505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 547/1087] 10.222.0.32 () {56 vars in 1120 bytes} [Mon May 30 11:51:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:56.917817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811c65d-65d2-4fe8-97a2-1589b7abc027 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:56.941555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.020074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811c65d-65d2-4fe8-97a2-1589b7abc027 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.055725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0385-8f0d-439c-a100-46e0127ee1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.056430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0385-8f0d-439c-a100-46e0127ee1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.056716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0385-8f0d-439c-a100-46e0127ee1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.056993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0385-8f0d-439c-a100-46e0127ee1c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.057737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0385-8f0d-439c-a100-46e0127ee1c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:57.058049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0385-8f0d-439c-a100-46e0127ee1c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.059401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0385-8f0d-439c-a100-46e0127ee1c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:57.068430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 541/1088] 10.222.0.32 () {56 vars in 1115 bytes} [Mon May 30 11:51:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:57.073810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49616e84-95b8-4f03-895a-20760ccf7331 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.119193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.133625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6545-53a6-46f4-8602-e9f520306ac2 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.133923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6545-53a6-46f4-8602-e9f520306ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.134204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6545-53a6-46f4-8602-e9f520306ac2 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.135579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 542/1089] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:51:57.140596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbea207-04df-4e37-a5d3-685fa5e6ef81 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.140891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbea207-04df-4e37-a5d3-685fa5e6ef81 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.141183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbea207-04df-4e37-a5d3-685fa5e6ef81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.158784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3cbea207-04df-4e37-a5d3-685fa5e6ef81 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:51:57.160083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3cbea207-04df-4e37-a5d3-685fa5e6ef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:51:57.176025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49616e84-95b8-4f03-895a-20760ccf7331 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.209229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.210017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.210301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.210577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.221319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.222793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:57.230172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 543/1090] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:51:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:57.240802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:57.241732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:57.242636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:57.242636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:57.242636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:57.242636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:57.242636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 548/1091] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:57.247582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbea207-04df-4e37-a5d3-685fa5e6ef81 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.254423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.263518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.349383 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbea207-04df-4e37-a5d3-685fa5e6ef81 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.353993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.361405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18868b-4bc3-4096-99eb-a5340350e5f8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.383991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.384624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.384911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.385195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.385922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:57.386235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.387614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:57.397098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a406554-bcf3-4d75-a4c4-827c6be2a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.397978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a406554-bcf3-4d75-a4c4-827c6be2a01e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.398199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a406554-bcf3-4d75-a4c4-827c6be2a01e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.398481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a406554-bcf3-4d75-a4c4-827c6be2a01e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.399754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 544/1092] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:57.405244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.412131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a406554-bcf3-4d75-a4c4-827c6be2a01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3a406554-bcf3-4d75-a4c4-827c6be2a01e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:51:57.419819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:57.420793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:57.421648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 549/1093] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:57.427410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a406554-bcf3-4d75-a4c4-827c6be2a01e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.479952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.505047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dddbe6-f081-451d-9d02-ebb218d2a27c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.529853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a406554-bcf3-4d75-a4c4-827c6be2a01e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.544202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.545026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.545307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.545585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.553318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.554757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:57.565015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.565690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.565976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.566278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.567585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:57.567695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:51:57.568732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:57.569641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 30 11:51:57.570621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:57.571196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:57.571269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 545/1094] 10.222.0.32 () {56 vars in 1093 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:57.575612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.583096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 550/1095] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:57.585277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.606021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.682587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cbf3b-7864-493b-a7ce-4fe7076c3ccb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.692488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37379c08-4456-44ae-84d6-90d6a259814c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.717270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.717948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.718234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.718514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.719360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:57.719675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.721131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:57.722746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.727727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a611d9-2453-46c8-a2c0-d3708178eb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.728627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a611d9-2453-46c8-a2c0-d3708178eb74 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.728911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a611d9-2453-46c8-a2c0-d3708178eb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.729190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a611d9-2453-46c8-a2c0-d3708178eb74 None admin] PATH_INFO: `/v3/users/38a5a25ab50c47fbaeff865cfbb078c0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.730383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a611d9-2453-46c8-a2c0-d3708178eb74 None admin] RBAC: Authorizing `identity:get_user(user_id=38a5a25ab50c47fbaeff865cfbb078c0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.731818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a611d9-2453-46c8-a2c0-d3708178eb74 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:57.733483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 551/1096] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/users/38a5a25ab50c47fbaeff865cfbb078c0 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:57.739028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a611d9-2453-46c8-a2c0-d3708178eb74 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.739937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 546/1097] 10.222.0.32 () {56 vars in 1133 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:57.745296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:57.826427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.842082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a611d9-2453-46c8-a2c0-d3708178eb74 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.852604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1a4d9-682b-4a1e-b5a9-180a721f01ca None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:57.877302 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868ab7d-426f-4cc4-955b-9f7e6ab170bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.877960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ab7d-426f-4cc4-955b-9f7e6ab170bc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.878274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ab7d-426f-4cc4-955b-9f7e6ab170bc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.878552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ab7d-426f-4cc4-955b-9f7e6ab170bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.887070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddbd64-8874-4cbd-8410-ad1e458e3c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:57.887722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddbd64-8874-4cbd-8410-ad1e458e3c72 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:57.888004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddbd64-8874-4cbd-8410-ad1e458e3c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:57.888316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddbd64-8874-4cbd-8410-ad1e458e3c72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:57.948978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:57.949927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868ab7d-426f-4cc4-955b-9f7e6ab170bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.951566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868ab7d-426f-4cc4-955b-9f7e6ab170bc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:57.958947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ddbd64-8874-4cbd-8410-ad1e458e3c72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:57.960427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ddbd64-8874-4cbd-8410-ad1e458e3c72 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:58.037597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:58.054672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 552/1098] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 30 11:51:58.060426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868ab7d-426f-4cc4-955b-9f7e6ab170bc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:58.060586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 547/1099] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:51:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 30 11:51:58.065815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddbd64-8874-4cbd-8410-ad1e458e3c72 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:58.164621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:58.166081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868ab7d-426f-4cc4-955b-9f7e6ab170bc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:58.174066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddbd64-8874-4cbd-8410-ad1e458e3c72 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:58.206621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94417d5-05ac-494e-a748-101209b7ff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:58.207571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94417d5-05ac-494e-a748-101209b7ff61 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:58.207867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94417d5-05ac-494e-a748-101209b7ff61 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:58.208154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94417d5-05ac-494e-a748-101209b7ff61 None admin] PATH_INFO: `/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:58.209001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:58.209355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94417d5-05ac-494e-a748-101209b7ff61 None admin] RBAC: Authorizing `identity:get_project(project_id=7f8a844b23f1431bbfe6b04e06dcf944)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:58.209794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:58.210079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:58.210355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:58.210818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94417d5-05ac-494e-a748-101209b7ff61 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:58.212573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 553/1100] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:51:58 2022] GET /identity/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:58.219175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94417d5-05ac-494e-a748-101209b7ff61 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:58.220594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:58.222036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:51:58.240115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:58.241264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:58.242196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:51:58.242196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:51:58.242196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:51:58.242196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:51:58.242196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 548/1101] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:51:58.246877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:58.276786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:58.322050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94417d5-05ac-494e-a748-101209b7ff61 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:58.354061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb196ee6-319b-419e-95cf-48bd2a1abbe2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:58.366004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:58.366694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:58.366981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:58.367268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:58.368108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=7f8a844b23f1431bbfe6b04e06dcf944, user.id=38a5a25ab50c47fbaeff865cfbb078c0) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:58.368456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:58.371451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:58.378178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 554/1102] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:58 2022] GET /identity/v3/role_assignments?user.id=38a5a25ab50c47fbaeff865cfbb078c0&scope.project.id=7f8a844b23f1431bbfe6b04e06dcf944&role.id=4d21498572b14478a996ac538be5382e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:58.393045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:58.393739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:58.394028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:58.394314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:58.395735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:58.396213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:58.398835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:58.413892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 549/1103] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:51:58.419737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:58.545935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208b5a3-2382-49c6-a955-56d1f7a060a3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:58.567487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:58.589198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:58.590038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:58.590326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:58.590606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:58.599547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:58.601701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:51:58.620739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:51:58.621704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:51:58.622587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 555/1104] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:51:58.628370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:58.634126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:51:58.758171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe1120-ff08-4be6-a899-68c94c6018b9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:58.799016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb2f2-ec49-4287-abde-fac15178f6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:58.799606 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb2f2-ec49-4287-abde-fac15178f6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:58.799888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb2f2-ec49-4287-abde-fac15178f6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:58.800186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb2f2-ec49-4287-abde-fac15178f6a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:58.800990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cb2f2-ec49-4287-abde-fac15178f6a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:58.802033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cb2f2-ec49-4287-abde-fac15178f6a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:58.804141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cb2f2-ec49-4287-abde-fac15178f6a9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:58.821585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 550/1105] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:51:58.828360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:51:58.944593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566f978-4f07-4e39-9bb8-dc5eb06eee27 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:51:58.982016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b06a8-4bc4-4c81-9ebe-909f847ceb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:51:58.982671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b06a8-4bc4-4c81-9ebe-909f847ceb47 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:51:58.982949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b06a8-4bc4-4c81-9ebe-909f847ceb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:51:58.983224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b06a8-4bc4-4c81-9ebe-909f847ceb47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:51:58.984051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b06a8-4bc4-4c81-9ebe-909f847ceb47 None admin] RBAC: Adding query filter params (role.id=8de96040db1e4e6e8c527b555d4c885a, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=70dce2f0284847ab8f07f55d700d0f60) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:51:58.984391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b06a8-4bc4-4c81-9ebe-909f847ceb47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:51:58.985829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b06a8-4bc4-4c81-9ebe-909f847ceb47 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:51:58.993435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 556/1106] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:51:58 2022] GET /identity/v3/role_assignments?user.id=70dce2f0284847ab8f07f55d700d0f60&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=8de96040db1e4e6e8c527b555d4c885a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:00.854021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48916ef5-c910-4203-8990-635b0cccefe7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:00.854537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48916ef5-c910-4203-8990-635b0cccefe7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:00.854537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48916ef5-c910-4203-8990-635b0cccefe7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:00.855847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 551/1107] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:52:00.861268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb771-eb87-40fb-8bc0-a85df55dc4c7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:00.861661 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb771-eb87-40fb-8bc0-a85df55dc4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:00.861839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb771-eb87-40fb-8bc0-a85df55dc4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:00.878448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-650eb771-eb87-40fb-8bc0-a85df55dc4c7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:00.880276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-650eb771-eb87-40fb-8bc0-a85df55dc4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:00.916074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:00.947290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 557/1108] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:00.972748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48916ef5-c910-4203-8990-635b0cccefe7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:01.036415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:01.076576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48916ef5-c910-4203-8990-635b0cccefe7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:01.109545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7546f-3e96-455c-83c2-b775f7a94065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:01.110270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7546f-3e96-455c-83c2-b775f7a94065 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:01.110557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7546f-3e96-455c-83c2-b775f7a94065 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:01.110833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7546f-3e96-455c-83c2-b775f7a94065 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:01.112005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7546f-3e96-455c-83c2-b775f7a94065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:01.114188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7546f-3e96-455c-83c2-b775f7a94065 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:01.115866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 552/1109] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:52:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:01.121695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eb771-eb87-40fb-8bc0-a85df55dc4c7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:01.235642 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eb771-eb87-40fb-8bc0-a85df55dc4c7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:01.271702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5f402-f7e2-46b3-a976-33c629f493ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:01.272410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f402-f7e2-46b3-a976-33c629f493ce None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:01.272699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f402-f7e2-46b3-a976-33c629f493ce None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:01.272981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5f402-f7e2-46b3-a976-33c629f493ce None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:01.273696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5f402-f7e2-46b3-a976-33c629f493ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:01.275160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5f402-f7e2-46b3-a976-33c629f493ce None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:01.294828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 558/1110] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:52:01 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:52:03.600597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1d60-b018-44d6-a580-ed524fadb2ee None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:03.601068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1d60-b018-44d6-a580-ed524fadb2ee None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:03.601159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1d60-b018-44d6-a580-ed524fadb2ee None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:03.602508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 553/1111] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:03.607902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255315d9-4f61-4f03-9de6-d4bb5939c623 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:03.608346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255315d9-4f61-4f03-9de6-d4bb5939c623 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:03.608506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255315d9-4f61-4f03-9de6-d4bb5939c623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:03.625579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-255315d9-4f61-4f03-9de6-d4bb5939c623 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:03.626780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-255315d9-4f61-4f03-9de6-d4bb5939c623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:03.649379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:03.694681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 559/1112] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:03.718001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a1d60-b018-44d6-a580-ed524fadb2ee None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:03.819564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a1d60-b018-44d6-a580-ed524fadb2ee None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:03.848495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:03.853534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a477a-c9e1-40ca-a79d-0c96815184c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:03.854386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a477a-c9e1-40ca-a79d-0c96815184c3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:03.854667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a477a-c9e1-40ca-a79d-0c96815184c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:03.854937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a477a-c9e1-40ca-a79d-0c96815184c3 None admin] PATH_INFO: `/v3/roles/a511e28f566a49b4bacfbd0325aa44ed` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:03.856434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a477a-c9e1-40ca-a79d-0c96815184c3 None admin] RBAC: Authorizing `identity:get_role(role_id=a511e28f566a49b4bacfbd0325aa44ed)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:03.857791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a477a-c9e1-40ca-a79d-0c96815184c3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:03.859141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 554/1113] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:03 2022] GET /identity/v3/roles/a511e28f566a49b4bacfbd0325aa44ed => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:03.865178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255315d9-4f61-4f03-9de6-d4bb5939c623 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:03.965114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255315d9-4f61-4f03-9de6-d4bb5939c623 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:03.998614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b39d0d-277b-4f01-b59d-f4aa0f3f9e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:03.999483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b39d0d-277b-4f01-b59d-f4aa0f3f9e0e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:03.999765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b39d0d-277b-4f01-b59d-f4aa0f3f9e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:04.000041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b39d0d-277b-4f01-b59d-f4aa0f3f9e0e None admin] PATH_INFO: `/v3/users/d54fce4bb8e741929622b1d4c8739f46` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:04.013706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b39d0d-277b-4f01-b59d-f4aa0f3f9e0e None admin] RBAC: Authorizing `identity:get_user(user_id=d54fce4bb8e741929622b1d4c8739f46)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:04.015139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b39d0d-277b-4f01-b59d-f4aa0f3f9e0e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:04.017221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 560/1114] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:03 2022] GET /identity/v3/users/d54fce4bb8e741929622b1d4c8739f46 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:04.023161 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a477a-c9e1-40ca-a79d-0c96815184c3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:04.123626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a477a-c9e1-40ca-a79d-0c96815184c3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:04.158250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427fd16-d415-4faf-b9f8-2818dca271b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:04.158902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427fd16-d415-4faf-b9f8-2818dca271b1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:04.159183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427fd16-d415-4faf-b9f8-2818dca271b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:04.159461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427fd16-d415-4faf-b9f8-2818dca271b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:04.222850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:04.225835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427fd16-d415-4faf-b9f8-2818dca271b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:04.227509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427fd16-d415-4faf-b9f8-2818dca271b1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:04.325591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 555/1115] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:52:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:52:04.331501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b39d0d-277b-4f01-b59d-f4aa0f3f9e0e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:04.349880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:04.433911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b39d0d-277b-4f01-b59d-f4aa0f3f9e0e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:04.469373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b149e-3b34-4336-b3fb-e03a8f9d7a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:04.470234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b149e-3b34-4336-b3fb-e03a8f9d7a71 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:04.470524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b149e-3b34-4336-b3fb-e03a8f9d7a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:04.470802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b149e-3b34-4336-b3fb-e03a8f9d7a71 None admin] PATH_INFO: `/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:04.473814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b149e-3b34-4336-b3fb-e03a8f9d7a71 None admin] RBAC: Authorizing `identity:get_project(project_id=7f8a844b23f1431bbfe6b04e06dcf944)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:04.475301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b149e-3b34-4336-b3fb-e03a8f9d7a71 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:04.477067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 561/1116] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:04 2022] GET /identity/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:04.483208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427fd16-d415-4faf-b9f8-2818dca271b1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:04.583947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:04.587209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427fd16-d415-4faf-b9f8-2818dca271b1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:04.627239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed8089-df9f-4254-a405-3ff0b38831f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:04.627940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed8089-df9f-4254-a405-3ff0b38831f8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:04.628214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed8089-df9f-4254-a405-3ff0b38831f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:04.628500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed8089-df9f-4254-a405-3ff0b38831f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:04.629378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed8089-df9f-4254-a405-3ff0b38831f8 None admin] RBAC: Adding query filter params (role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=7f8a844b23f1431bbfe6b04e06dcf944, user.id=d54fce4bb8e741929622b1d4c8739f46) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:04.629693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed8089-df9f-4254-a405-3ff0b38831f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:04.631250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed8089-df9f-4254-a405-3ff0b38831f8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:04.637445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 556/1117] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:52:04 2022] GET /identity/v3/role_assignments?user.id=d54fce4bb8e741929622b1d4c8739f46&scope.project.id=7f8a844b23f1431bbfe6b04e06dcf944&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:06.846961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c8543-99d9-4204-86ad-f2ea5703d498 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:06.847473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c8543-99d9-4204-86ad-f2ea5703d498 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:06.847473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c8543-99d9-4204-86ad-f2ea5703d498 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:06.849380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 562/1118] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:06.854793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a665dcc-4b4d-4c3e-9cf9-f791348e4eaa None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:06.855202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a665dcc-4b4d-4c3e-9cf9-f791348e4eaa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:06.855341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a665dcc-4b4d-4c3e-9cf9-f791348e4eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:06.872581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7a665dcc-4b4d-4c3e-9cf9-f791348e4eaa None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:06.873749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a665dcc-4b4d-4c3e-9cf9-f791348e4eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:06.942774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 557/1119] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:06.965334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c8543-99d9-4204-86ad-f2ea5703d498 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:07.049872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:07.066597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c8543-99d9-4204-86ad-f2ea5703d498 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:07.100404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e3b21-d211-4ff8-bf51-129cdcfcf50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:07.101195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3b21-d211-4ff8-bf51-129cdcfcf50d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:07.101482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3b21-d211-4ff8-bf51-129cdcfcf50d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:07.101761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3b21-d211-4ff8-bf51-129cdcfcf50d None admin] PATH_INFO: `/v3/roles/a511e28f566a49b4bacfbd0325aa44ed` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:07.103230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e3b21-d211-4ff8-bf51-129cdcfcf50d None admin] RBAC: Authorizing `identity:get_role(role_id=a511e28f566a49b4bacfbd0325aa44ed)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:07.104645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e3b21-d211-4ff8-bf51-129cdcfcf50d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:07.106023 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 563/1120] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:06 2022] GET /identity/v3/roles/a511e28f566a49b4bacfbd0325aa44ed => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:07.111863 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a665dcc-4b4d-4c3e-9cf9-f791348e4eaa None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:07.201642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:07.209206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a665dcc-4b4d-4c3e-9cf9-f791348e4eaa None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:07.243644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d3c8b-0da6-4360-93de-93ac52bbc06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:07.244540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d3c8b-0da6-4360-93de-93ac52bbc06c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:07.244827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d3c8b-0da6-4360-93de-93ac52bbc06c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:07.245105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d3c8b-0da6-4360-93de-93ac52bbc06c None admin] PATH_INFO: `/v3/users/d54fce4bb8e741929622b1d4c8739f46` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:07.246292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d3c8b-0da6-4360-93de-93ac52bbc06c None admin] RBAC: Authorizing `identity:get_user(user_id=d54fce4bb8e741929622b1d4c8739f46)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:07.247728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d3c8b-0da6-4360-93de-93ac52bbc06c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:07.249434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 558/1121] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:07 2022] GET /identity/v3/users/d54fce4bb8e741929622b1d4c8739f46 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:07.255784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e3b21-d211-4ff8-bf51-129cdcfcf50d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:07.353404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e3b21-d211-4ff8-bf51-129cdcfcf50d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:07.387593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0471035-8000-4e34-af55-b5e40549139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:07.388482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471035-8000-4e34-af55-b5e40549139c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:07.388771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471035-8000-4e34-af55-b5e40549139c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:07.389051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471035-8000-4e34-af55-b5e40549139c None admin] PATH_INFO: `/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:07.390225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0471035-8000-4e34-af55-b5e40549139c None admin] RBAC: Authorizing `identity:get_project(project_id=7f8a844b23f1431bbfe6b04e06dcf944)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:07.391954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0471035-8000-4e34-af55-b5e40549139c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:07.393684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 564/1122] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:07 2022] GET /identity/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:07.399668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d3c8b-0da6-4360-93de-93ac52bbc06c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:07.498400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d3c8b-0da6-4360-93de-93ac52bbc06c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:07.531187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee1d8e-8312-4062-b7dd-33825d67fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:07.532064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee1d8e-8312-4062-b7dd-33825d67fe02 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:07.532370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee1d8e-8312-4062-b7dd-33825d67fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:07.532647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee1d8e-8312-4062-b7dd-33825d67fe02 None admin] PATH_INFO: `/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944/users/d54fce4bb8e741929622b1d4c8739f46/roles/a511e28f566a49b4bacfbd0325aa44ed` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:07.534799 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee1d8e-8312-4062-b7dd-33825d67fe02 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f8a844b23f1431bbfe6b04e06dcf944, user_id=d54fce4bb8e741929622b1d4c8739f46, role_id=a511e28f566a49b4bacfbd0325aa44ed)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:07.537018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee1d8e-8312-4062-b7dd-33825d67fe02 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:07.549109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 559/1123] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:52:07 2022] PUT /identity/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944/users/d54fce4bb8e741929622b1d4c8739f46/roles/a511e28f566a49b4bacfbd0325aa44ed => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:52:09.815005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db64c4c-0829-4d90-a052-20478a3f7813 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:09.815573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db64c4c-0829-4d90-a052-20478a3f7813 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:09.815573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db64c4c-0829-4d90-a052-20478a3f7813 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:09.817677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 565/1124] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:09.823619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865968ca-41a2-470b-8cce-4595d971d108 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:09.824100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865968ca-41a2-470b-8cce-4595d971d108 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:09.824404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865968ca-41a2-470b-8cce-4595d971d108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:09.841976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-865968ca-41a2-470b-8cce-4595d971d108 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:09.843095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-865968ca-41a2-470b-8cce-4595d971d108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:09.859189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:09.910952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 560/1125] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:09.934784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db64c4c-0829-4d90-a052-20478a3f7813 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:09.956987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:10.040803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db64c4c-0829-4d90-a052-20478a3f7813 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:10.075575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e6b86-b256-48bf-b883-7a4658ffb0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:10.076437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e6b86-b256-48bf-b883-7a4658ffb0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:10.076722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e6b86-b256-48bf-b883-7a4658ffb0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:10.076999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e6b86-b256-48bf-b883-7a4658ffb0b8 None admin] PATH_INFO: `/v3/roles/a511e28f566a49b4bacfbd0325aa44ed` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:10.078443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e6b86-b256-48bf-b883-7a4658ffb0b8 None admin] RBAC: Authorizing `identity:get_role(role_id=a511e28f566a49b4bacfbd0325aa44ed)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:10.079803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e6b86-b256-48bf-b883-7a4658ffb0b8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:10.081198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 566/1126] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:09 2022] GET /identity/v3/roles/a511e28f566a49b4bacfbd0325aa44ed => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:10.087645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865968ca-41a2-470b-8cce-4595d971d108 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:10.188868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865968ca-41a2-470b-8cce-4595d971d108 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:10.225971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae6a47-6bd4-4c90-8e35-d2d554750df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:10.226857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6a47-6bd4-4c90-8e35-d2d554750df1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:10.227145 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6a47-6bd4-4c90-8e35-d2d554750df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:10.227421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6a47-6bd4-4c90-8e35-d2d554750df1 None admin] PATH_INFO: `/v3/users/d54fce4bb8e741929622b1d4c8739f46` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:10.228649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae6a47-6bd4-4c90-8e35-d2d554750df1 None admin] RBAC: Authorizing `identity:get_user(user_id=d54fce4bb8e741929622b1d4c8739f46)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:10.230095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae6a47-6bd4-4c90-8e35-d2d554750df1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:10.231839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 561/1127] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:10 2022] GET /identity/v3/users/d54fce4bb8e741929622b1d4c8739f46 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:10.237861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e6b86-b256-48bf-b883-7a4658ffb0b8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:10.326708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:10.339113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e6b86-b256-48bf-b883-7a4658ffb0b8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:10.375021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a242c8-d323-45b8-b78b-4315a96af159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:10.375672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a242c8-d323-45b8-b78b-4315a96af159 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:10.375957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a242c8-d323-45b8-b78b-4315a96af159 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:10.376263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a242c8-d323-45b8-b78b-4315a96af159 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:10.443479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a242c8-d323-45b8-b78b-4315a96af159 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:10.444887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a242c8-d323-45b8-b78b-4315a96af159 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:10.533531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:10.546740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 567/1128] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:52:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:52:10.553581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae6a47-6bd4-4c90-8e35-d2d554750df1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:10.575069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:10.652494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae6a47-6bd4-4c90-8e35-d2d554750df1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:10.686108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168b046-d92e-4b98-917d-2518c0c5ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:10.686961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168b046-d92e-4b98-917d-2518c0c5ed27 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:10.687243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168b046-d92e-4b98-917d-2518c0c5ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:10.687517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168b046-d92e-4b98-917d-2518c0c5ed27 None admin] PATH_INFO: `/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:10.688703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168b046-d92e-4b98-917d-2518c0c5ed27 None admin] RBAC: Authorizing `identity:get_project(project_id=7f8a844b23f1431bbfe6b04e06dcf944)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:10.690125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168b046-d92e-4b98-917d-2518c0c5ed27 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:10.692180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 562/1129] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:10 2022] GET /identity/v3/projects/7f8a844b23f1431bbfe6b04e06dcf944 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:10.698549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a242c8-d323-45b8-b78b-4315a96af159 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:10.796865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a242c8-d323-45b8-b78b-4315a96af159 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:10.830362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1805718-6038-4624-9bdd-19e6cc4b96f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:10.831019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1805718-6038-4624-9bdd-19e6cc4b96f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:10.831308 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1805718-6038-4624-9bdd-19e6cc4b96f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:10.831589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1805718-6038-4624-9bdd-19e6cc4b96f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:10.832982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1805718-6038-4624-9bdd-19e6cc4b96f3 None admin] RBAC: Adding query filter params (role.id=a511e28f566a49b4bacfbd0325aa44ed, scope.project.id=7f8a844b23f1431bbfe6b04e06dcf944, user.id=d54fce4bb8e741929622b1d4c8739f46) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:10.833332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1805718-6038-4624-9bdd-19e6cc4b96f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:10.834785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1805718-6038-4624-9bdd-19e6cc4b96f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:10.841236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 568/1130] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:52:10 2022] GET /identity/v3/role_assignments?user.id=d54fce4bb8e741929622b1d4c8739f46&scope.project.id=7f8a844b23f1431bbfe6b04e06dcf944&role.id=a511e28f566a49b4bacfbd0325aa44ed => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:13.126020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebefb0-ce34-49fa-b172-bd5f08d48a9d None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:13.126610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebefb0-ce34-49fa-b172-bd5f08d48a9d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:13.126610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebefb0-ce34-49fa-b172-bd5f08d48a9d None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:13.127954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 563/1131] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:13.133850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eeff9-1763-4014-ba67-6a886969eef7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:13.134267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eeff9-1763-4014-ba67-6a886969eef7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:13.134410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eeff9-1763-4014-ba67-6a886969eef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:13.151784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-176eeff9-1763-4014-ba67-6a886969eef7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:13.152984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-176eeff9-1763-4014-ba67-6a886969eef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:13.198497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:13.218471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 569/1132] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:13.247070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebefb0-ce34-49fa-b172-bd5f08d48a9d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:13.336603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:13.350747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebefb0-ce34-49fa-b172-bd5f08d48a9d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:13.384582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3bff5-8a68-481f-a055-e9f65d604369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:13.385299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3bff5-8a68-481f-a055-e9f65d604369 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:13.385584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3bff5-8a68-481f-a055-e9f65d604369 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:13.385862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3bff5-8a68-481f-a055-e9f65d604369 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:13.387111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3bff5-8a68-481f-a055-e9f65d604369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:13.388569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3bff5-8a68-481f-a055-e9f65d604369 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:13.390175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 564/1133] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:52:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:13.397212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eeff9-1763-4014-ba67-6a886969eef7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:13.494211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eeff9-1763-4014-ba67-6a886969eef7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:13.533652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938713e4-2ba5-4fde-9806-7684050b4a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:13.534339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938713e4-2ba5-4fde-9806-7684050b4a43 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:13.535844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938713e4-2ba5-4fde-9806-7684050b4a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:13.536133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938713e4-2ba5-4fde-9806-7684050b4a43 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:13.536905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938713e4-2ba5-4fde-9806-7684050b4a43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:13.540580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938713e4-2ba5-4fde-9806-7684050b4a43 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:13.569671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 570/1134] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:52:13 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:52:15.875665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5090bee-9d81-4906-9371-2d9cc1985188 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:15.876198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5090bee-9d81-4906-9371-2d9cc1985188 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:15.876198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5090bee-9d81-4906-9371-2d9cc1985188 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:15.877539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 565/1135] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:15.883730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7d24f-4450-4258-a7dd-038722feef35 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:15.884320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7d24f-4450-4258-a7dd-038722feef35 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:15.884320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7d24f-4450-4258-a7dd-038722feef35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:15.902499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4bc7d24f-4450-4258-a7dd-038722feef35 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:15.903711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4bc7d24f-4450-4258-a7dd-038722feef35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:15.931197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:15.971550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 571/1136] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:15.997005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5090bee-9d81-4906-9371-2d9cc1985188 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:16.097169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5090bee-9d81-4906-9371-2d9cc1985188 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:16.132943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c1597-79f1-439e-a66b-4182e749d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:16.133661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1597-79f1-439e-a66b-4182e749d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:16.133948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1597-79f1-439e-a66b-4182e749d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:16.134228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1597-79f1-439e-a66b-4182e749d6a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:16.135383 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c1597-79f1-439e-a66b-4182e749d6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:16.137001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c1597-79f1-439e-a66b-4182e749d6a5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:16.138613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 566/1137] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:52:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:16.144669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7d24f-4450-4258-a7dd-038722feef35 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:16.249040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7d24f-4450-4258-a7dd-038722feef35 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:16.291356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d9275-35c2-4e88-ad4a-514488cc17e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:16.292039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9275-35c2-4e88-ad4a-514488cc17e0 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:16.292351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9275-35c2-4e88-ad4a-514488cc17e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:16.292635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9275-35c2-4e88-ad4a-514488cc17e0 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:16.293360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d9275-35c2-4e88-ad4a-514488cc17e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:16.294999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d9275-35c2-4e88-ad4a-514488cc17e0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:16.317900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 572/1138] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:52:16 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:52:18.633567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f663c30-aece-4be3-914b-7331a0263776 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:18.634133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f663c30-aece-4be3-914b-7331a0263776 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:18.634234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f663c30-aece-4be3-914b-7331a0263776 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:18.635594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 567/1139] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:18.640961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9646081-acd1-44b9-b555-4413780b84b9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:18.641380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9646081-acd1-44b9-b555-4413780b84b9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:18.641783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9646081-acd1-44b9-b555-4413780b84b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:18.659070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c9646081-acd1-44b9-b555-4413780b84b9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:18.660330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c9646081-acd1-44b9-b555-4413780b84b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:18.674014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:18.726700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 573/1140] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:18.750576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f663c30-aece-4be3-914b-7331a0263776 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:18.777067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:18.851616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f663c30-aece-4be3-914b-7331a0263776 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:18.884880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d763d0-af45-46de-9d21-90331267cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:18.885679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d763d0-af45-46de-9d21-90331267cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:18.885961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d763d0-af45-46de-9d21-90331267cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:18.886235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d763d0-af45-46de-9d21-90331267cfe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:18.897307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d763d0-af45-46de-9d21-90331267cfe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-21d763d0-af45-46de-9d21-90331267cfe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:52:18.904871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:52:18.906066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:52:18.907343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 568/1141] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:52:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:52:18.912520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9646081-acd1-44b9-b555-4413780b84b9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:19.010834 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9646081-acd1-44b9-b555-4413780b84b9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:19.043968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:19.044698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:19.044986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:19.045268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:19.046020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:19.046335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:19.047705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:19.056363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 574/1142] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:52:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:19.062124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d763d0-af45-46de-9d21-90331267cfe7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:19.146416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:19.162141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d763d0-af45-46de-9d21-90331267cfe7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:19.195419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850cb6b-96ac-4b0b-94ba-96bcb998c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:19.196309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850cb6b-96ac-4b0b-94ba-96bcb998c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:19.196598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850cb6b-96ac-4b0b-94ba-96bcb998c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:19.196876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850cb6b-96ac-4b0b-94ba-96bcb998c3b6 None admin] PATH_INFO: `/v3/users/c56b002edfa54466b995a71149bbf7b1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:19.209730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850cb6b-96ac-4b0b-94ba-96bcb998c3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=c56b002edfa54466b995a71149bbf7b1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:19.211179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850cb6b-96ac-4b0b-94ba-96bcb998c3b6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:19.212846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 569/1143] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:19 2022] GET /identity/v3/users/c56b002edfa54466b995a71149bbf7b1 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:19.218623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:19.233535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:19.321904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7b6d3-7270-4be4-b4e9-276a7c41c036 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:19.354460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7138-2510-4592-9e1d-8af9e9eb4023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:19.355099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7138-2510-4592-9e1d-8af9e9eb4023 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:19.355381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7138-2510-4592-9e1d-8af9e9eb4023 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:19.355657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d7138-2510-4592-9e1d-8af9e9eb4023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:19.422510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d7138-2510-4592-9e1d-8af9e9eb4023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:19.423915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d7138-2510-4592-9e1d-8af9e9eb4023 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:19.438005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:19.520896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 575/1144] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:52:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:52:19.526733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850cb6b-96ac-4b0b-94ba-96bcb998c3b6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:19.625108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850cb6b-96ac-4b0b-94ba-96bcb998c3b6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:19.658170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7bef7-e804-4b06-861b-eed886be95be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:19.659037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7bef7-e804-4b06-861b-eed886be95be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:19.659322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7bef7-e804-4b06-861b-eed886be95be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:19.659601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7bef7-e804-4b06-861b-eed886be95be None admin] PATH_INFO: `/v3/projects/71171cbb25ec43c8989c6a98c27fa1de` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:19.661560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7bef7-e804-4b06-861b-eed886be95be None admin] RBAC: Authorizing `identity:get_project(project_id=71171cbb25ec43c8989c6a98c27fa1de)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:19.662995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7bef7-e804-4b06-861b-eed886be95be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:19.664734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 570/1145] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:19 2022] GET /identity/v3/projects/71171cbb25ec43c8989c6a98c27fa1de => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:19.670994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7138-2510-4592-9e1d-8af9e9eb4023 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:19.767911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d7138-2510-4592-9e1d-8af9e9eb4023 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:19.782119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:19.803953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bb8f2-87c2-48fa-a79b-46edc797fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:19.804641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bb8f2-87c2-48fa-a79b-46edc797fcea None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:19.804937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bb8f2-87c2-48fa-a79b-46edc797fcea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:19.805202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bb8f2-87c2-48fa-a79b-46edc797fcea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:19.806023 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bb8f2-87c2-48fa-a79b-46edc797fcea None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=71171cbb25ec43c8989c6a98c27fa1de, user.id=c56b002edfa54466b995a71149bbf7b1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:19.806338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bb8f2-87c2-48fa-a79b-46edc797fcea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:19.807762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bb8f2-87c2-48fa-a79b-46edc797fcea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:19.813245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 576/1146] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:52:19 2022] GET /identity/v3/role_assignments?user.id=c56b002edfa54466b995a71149bbf7b1&scope.project.id=71171cbb25ec43c8989c6a98c27fa1de&role.id=4d21498572b14478a996ac538be5382e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:22.023317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716abec0-5f6c-46c3-8fac-0d0e5b949d52 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:22.023816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716abec0-5f6c-46c3-8fac-0d0e5b949d52 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:22.023816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716abec0-5f6c-46c3-8fac-0d0e5b949d52 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:22.025180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 571/1147] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:22.030402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a718c5-d455-4552-be8d-9effe1e542a9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:22.030846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a718c5-d455-4552-be8d-9effe1e542a9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:22.030975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a718c5-d455-4552-be8d-9effe1e542a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:22.048098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f6a718c5-d455-4552-be8d-9effe1e542a9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:22.049315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f6a718c5-d455-4552-be8d-9effe1e542a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:22.113668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 577/1148] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:22.135987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716abec0-5f6c-46c3-8fac-0d0e5b949d52 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:22.148971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:22.237521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716abec0-5f6c-46c3-8fac-0d0e5b949d52 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:22.271994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a78b53-bc32-4b30-86d3-74f51d4bf247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:22.272806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a78b53-bc32-4b30-86d3-74f51d4bf247 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:22.273091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a78b53-bc32-4b30-86d3-74f51d4bf247 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:22.273415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a78b53-bc32-4b30-86d3-74f51d4bf247 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:22.283229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a78b53-bc32-4b30-86d3-74f51d4bf247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-55a78b53-bc32-4b30-86d3-74f51d4bf247 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:52:22.290865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:52:22.291984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:52:22.292918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 572/1149] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:52:22.298145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a718c5-d455-4552-be8d-9effe1e542a9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:22.399718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a718c5-d455-4552-be8d-9effe1e542a9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:22.406836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:22.433156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:22.433821 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:22.434108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:22.434386 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:22.435127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:22.435437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:22.436848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:22.445335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 578/1150] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:52:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:22.451487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a78b53-bc32-4b30-86d3-74f51d4bf247 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:22.530402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:22.563011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a78b53-bc32-4b30-86d3-74f51d4bf247 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:22.597529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74c7a9-1e42-4bbe-9f09-b4b1bfbe3ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:22.598412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74c7a9-1e42-4bbe-9f09-b4b1bfbe3ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:22.598692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74c7a9-1e42-4bbe-9f09-b4b1bfbe3ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:22.598967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74c7a9-1e42-4bbe-9f09-b4b1bfbe3ed2 None admin] PATH_INFO: `/v3/users/c56b002edfa54466b995a71149bbf7b1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:22.600505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74c7a9-1e42-4bbe-9f09-b4b1bfbe3ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=c56b002edfa54466b995a71149bbf7b1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:22.601945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74c7a9-1e42-4bbe-9f09-b4b1bfbe3ed2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:22.603573 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 573/1151] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:22 2022] GET /identity/v3/users/c56b002edfa54466b995a71149bbf7b1 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:22.609286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:22.712767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63eb73c-7a73-4c04-bdb7-807c31c41bcb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:22.745565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a6874-bd37-407f-a499-c555782f0f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:22.746425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a6874-bd37-407f-a499-c555782f0f73 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:22.746716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a6874-bd37-407f-a499-c555782f0f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:22.747001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a6874-bd37-407f-a499-c555782f0f73 None admin] PATH_INFO: `/v3/projects/71171cbb25ec43c8989c6a98c27fa1de` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:22.748206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a6874-bd37-407f-a499-c555782f0f73 None admin] RBAC: Authorizing `identity:get_project(project_id=71171cbb25ec43c8989c6a98c27fa1de)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:22.749646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a6874-bd37-407f-a499-c555782f0f73 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:22.751411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 579/1152] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:22 2022] GET /identity/v3/projects/71171cbb25ec43c8989c6a98c27fa1de => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:22.758134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74c7a9-1e42-4bbe-9f09-b4b1bfbe3ed2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:22.857300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74c7a9-1e42-4bbe-9f09-b4b1bfbe3ed2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:22.890385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690589b8-b68b-4cd1-8c2a-04ee3a25c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:22.891275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690589b8-b68b-4cd1-8c2a-04ee3a25c075 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:22.891812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690589b8-b68b-4cd1-8c2a-04ee3a25c075 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:22.892095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690589b8-b68b-4cd1-8c2a-04ee3a25c075 None admin] PATH_INFO: `/v3/projects/71171cbb25ec43c8989c6a98c27fa1de/users/c56b002edfa54466b995a71149bbf7b1/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:22.894679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690589b8-b68b-4cd1-8c2a-04ee3a25c075 None admin] RBAC: Authorizing `identity:create_grant(project_id=71171cbb25ec43c8989c6a98c27fa1de, user_id=c56b002edfa54466b995a71149bbf7b1, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:22.896219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690589b8-b68b-4cd1-8c2a-04ee3a25c075 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:22.905113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:22.911939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 574/1153] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:52:22 2022] PUT /identity/v3/projects/71171cbb25ec43c8989c6a98c27fa1de/users/c56b002edfa54466b995a71149bbf7b1/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:52:25.159540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481e04-7cd6-42ac-b297-bf677caae7b8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:25.160256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481e04-7cd6-42ac-b297-bf677caae7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:25.160256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481e04-7cd6-42ac-b297-bf677caae7b8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:25.161520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 580/1154] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:25.166688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252253a8-9334-44c1-82f8-aacde2edead5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:25.167236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252253a8-9334-44c1-82f8-aacde2edead5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:25.167236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252253a8-9334-44c1-82f8-aacde2edead5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:25.184545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-252253a8-9334-44c1-82f8-aacde2edead5 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:25.185729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-252253a8-9334-44c1-82f8-aacde2edead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:25.251025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 575/1155] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:25.276320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481e04-7cd6-42ac-b297-bf677caae7b8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:25.312077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:25.379566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19481e04-7cd6-42ac-b297-bf677caae7b8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:25.412497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e0424-ef4f-455d-b883-ebfe381835d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:25.413217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e0424-ef4f-455d-b883-ebfe381835d0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:25.413503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e0424-ef4f-455d-b883-ebfe381835d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:25.413782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e0424-ef4f-455d-b883-ebfe381835d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:25.424129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e0424-ef4f-455d-b883-ebfe381835d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3b9e0424-ef4f-455d-b883-ebfe381835d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:52:25.432401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:52:25.433481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:52:25.434669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 581/1156] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:52:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:52:25.439639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252253a8-9334-44c1-82f8-aacde2edead5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:25.538828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252253a8-9334-44c1-82f8-aacde2edead5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:25.564526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:25.573142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:25.573819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:25.574122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:25.574400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:25.575153 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:25.575466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:25.576883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:25.585516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 576/1157] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:52:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:25.591220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e0424-ef4f-455d-b883-ebfe381835d0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:25.685505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:25.692137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e0424-ef4f-455d-b883-ebfe381835d0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:25.726250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e2c49-f31f-40ba-8e53-23f0467823f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:25.727120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2c49-f31f-40ba-8e53-23f0467823f5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:25.727407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2c49-f31f-40ba-8e53-23f0467823f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:25.727685 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2c49-f31f-40ba-8e53-23f0467823f5 None admin] PATH_INFO: `/v3/users/c56b002edfa54466b995a71149bbf7b1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:25.728909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e2c49-f31f-40ba-8e53-23f0467823f5 None admin] RBAC: Authorizing `identity:get_user(user_id=c56b002edfa54466b995a71149bbf7b1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:25.730363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e2c49-f31f-40ba-8e53-23f0467823f5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:25.731991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 582/1158] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:25 2022] GET /identity/v3/users/c56b002edfa54466b995a71149bbf7b1 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:25.738549 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:25.837960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edd914-1635-46ec-bb56-b7717c29e03d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:25.870675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949cc03-f72b-4612-8d46-a4acdb1d0b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:25.871317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949cc03-f72b-4612-8d46-a4acdb1d0b57 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:25.871601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949cc03-f72b-4612-8d46-a4acdb1d0b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:25.871879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949cc03-f72b-4612-8d46-a4acdb1d0b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:25.929963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:25.940701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949cc03-f72b-4612-8d46-a4acdb1d0b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:25.942279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949cc03-f72b-4612-8d46-a4acdb1d0b57 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:26.037872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 577/1159] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:52:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:52:26.043748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e2c49-f31f-40ba-8e53-23f0467823f5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:26.144814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e2c49-f31f-40ba-8e53-23f0467823f5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:26.178522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7373f85-d799-4d4e-bb7e-ef6be6d91850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:26.179381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7373f85-d799-4d4e-bb7e-ef6be6d91850 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:26.179671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7373f85-d799-4d4e-bb7e-ef6be6d91850 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:26.179950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7373f85-d799-4d4e-bb7e-ef6be6d91850 None admin] PATH_INFO: `/v3/projects/71171cbb25ec43c8989c6a98c27fa1de` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:26.181166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7373f85-d799-4d4e-bb7e-ef6be6d91850 None admin] RBAC: Authorizing `identity:get_project(project_id=71171cbb25ec43c8989c6a98c27fa1de)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:26.182643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7373f85-d799-4d4e-bb7e-ef6be6d91850 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:26.184412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 583/1160] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:26 2022] GET /identity/v3/projects/71171cbb25ec43c8989c6a98c27fa1de => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:26.190583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949cc03-f72b-4612-8d46-a4acdb1d0b57 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:26.278559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:26.289269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949cc03-f72b-4612-8d46-a4acdb1d0b57 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:26.323825 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefc101-da58-49bd-9634-cae5822a493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:26.324509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc101-da58-49bd-9634-cae5822a493f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:26.324800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc101-da58-49bd-9634-cae5822a493f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:26.325075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc101-da58-49bd-9634-cae5822a493f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:26.325901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc101-da58-49bd-9634-cae5822a493f None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=71171cbb25ec43c8989c6a98c27fa1de, user.id=c56b002edfa54466b995a71149bbf7b1) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:26.326215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc101-da58-49bd-9634-cae5822a493f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:26.327649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc101-da58-49bd-9634-cae5822a493f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:26.333940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 578/1161] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:52:26 2022] GET /identity/v3/role_assignments?user.id=c56b002edfa54466b995a71149bbf7b1&scope.project.id=71171cbb25ec43c8989c6a98c27fa1de&role.id=4d21498572b14478a996ac538be5382e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:28.619312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec02a5-1031-423f-b2f4-396101d00abd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:28.619816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec02a5-1031-423f-b2f4-396101d00abd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:28.619909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec02a5-1031-423f-b2f4-396101d00abd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:28.621293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 584/1162] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:28.626560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e394bc-3e9c-46b7-87bf-e14c52e4c2cd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:28.626996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e394bc-3e9c-46b7-87bf-e14c52e4c2cd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:28.627129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e394bc-3e9c-46b7-87bf-e14c52e4c2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:28.646719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-71e394bc-3e9c-46b7-87bf-e14c52e4c2cd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:28.647907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-71e394bc-3e9c-46b7-87bf-e14c52e4c2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:28.717652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 579/1163] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:28.741156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec02a5-1031-423f-b2f4-396101d00abd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:28.772859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:28.842988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec02a5-1031-423f-b2f4-396101d00abd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:28.876286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc616b-f465-4d2e-bad9-d325c4141715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:28.876936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc616b-f465-4d2e-bad9-d325c4141715 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:28.877229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc616b-f465-4d2e-bad9-d325c4141715 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:28.877520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc616b-f465-4d2e-bad9-d325c4141715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:28.945168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc616b-f465-4d2e-bad9-d325c4141715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:28.946635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc616b-f465-4d2e-bad9-d325c4141715 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:28.978549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:29.050849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 585/1164] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:52:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:52:29.057082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e394bc-3e9c-46b7-87bf-e14c52e4c2cd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:29.140581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:29.156734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e394bc-3e9c-46b7-87bf-e14c52e4c2cd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:29.190534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1ba56-0317-4955-a675-7c0bdc9eae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:29.191339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1ba56-0317-4955-a675-7c0bdc9eae95 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:29.191633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1ba56-0317-4955-a675-7c0bdc9eae95 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:29.191916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1ba56-0317-4955-a675-7c0bdc9eae95 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:29.198500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1ba56-0317-4955-a675-7c0bdc9eae95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:29.199941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1ba56-0317-4955-a675-7c0bdc9eae95 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8ed1ba56-0317-4955-a675-7c0bdc9eae95 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:52:29.213944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:52:29.215095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:52:29.216002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 580/1165] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:52:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:52:29.221287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc616b-f465-4d2e-bad9-d325c4141715 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:29.319397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc616b-f465-4d2e-bad9-d325c4141715 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:29.355097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:29.360777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c149893-d94f-4dc9-9e67-e4eaf23127a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:29.361455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c149893-d94f-4dc9-9e67-e4eaf23127a3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:29.361743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c149893-d94f-4dc9-9e67-e4eaf23127a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:29.362022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c149893-d94f-4dc9-9e67-e4eaf23127a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:29.362752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c149893-d94f-4dc9-9e67-e4eaf23127a3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:29.363067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c149893-d94f-4dc9-9e67-e4eaf23127a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:29.364494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c149893-d94f-4dc9-9e67-e4eaf23127a3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:29.371962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 586/1166] 10.222.0.32 () {56 vars in 1115 bytes} [Mon May 30 11:52:29 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:31.623591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb88204-a860-43a7-8ea0-892bdbdd6eb0 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:31.624541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb88204-a860-43a7-8ea0-892bdbdd6eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:31.624836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb88204-a860-43a7-8ea0-892bdbdd6eb0 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:31.626250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 581/1167] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:31.632463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e1f91-2daf-4b10-9c8e-2355e8232d77 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:31.632879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e1f91-2daf-4b10-9c8e-2355e8232d77 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:31.633019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e1f91-2daf-4b10-9c8e-2355e8232d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:31.650602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e32e1f91-2daf-4b10-9c8e-2355e8232d77 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:31.651767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e32e1f91-2daf-4b10-9c8e-2355e8232d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:31.719336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 587/1168] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:31.744690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb88204-a860-43a7-8ea0-892bdbdd6eb0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:31.848062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb88204-a860-43a7-8ea0-892bdbdd6eb0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:31.881091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd7130-ad6c-4ec1-a8bf-07ab5301f2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:31.881755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7130-ad6c-4ec1-a8bf-07ab5301f2be None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:31.882036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7130-ad6c-4ec1-a8bf-07ab5301f2be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:31.882309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7130-ad6c-4ec1-a8bf-07ab5301f2be None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:31.883059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd7130-ad6c-4ec1-a8bf-07ab5301f2be None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:31.884444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd7130-ad6c-4ec1-a8bf-07ab5301f2be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:31.900130 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 582/1169] 10.222.0.32 () {60 vars in 1138 bytes} [Mon May 30 11:52:31 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:52:34.211591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfdb1c-7bbb-4cf2-96cf-d97f612c58bd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:34.212123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfdb1c-7bbb-4cf2-96cf-d97f612c58bd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:34.212123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfdb1c-7bbb-4cf2-96cf-d97f612c58bd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:34.213552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 588/1170] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:34.218811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ea1d4-3d61-4b66-840f-d62e26e76dce None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:34.219234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ea1d4-3d61-4b66-840f-d62e26e76dce None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:34.219378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ea1d4-3d61-4b66-840f-d62e26e76dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:34.225948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:34.238183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9a0ea1d4-3d61-4b66-840f-d62e26e76dce None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:34.239379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9a0ea1d4-3d61-4b66-840f-d62e26e76dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:34.305455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 583/1171] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:34.328680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfdb1c-7bbb-4cf2-96cf-d97f612c58bd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:34.430162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfdb1c-7bbb-4cf2-96cf-d97f612c58bd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:34.457199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:34.465977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f75577-c4c9-4350-9663-9d692c907ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:34.466780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75577-c4c9-4350-9663-9d692c907ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:34.467064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75577-c4c9-4350-9663-9d692c907ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:34.467339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75577-c4c9-4350-9663-9d692c907ed0 None admin] PATH_INFO: `/v3/domains/cfedc9def0e84875b9915ef0717522d3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:34.478025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f75577-c4c9-4350-9663-9d692c907ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cfedc9def0e84875b9915ef0717522d3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:34.479464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f75577-c4c9-4350-9663-9d692c907ed0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:34.481112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 589/1172] 10.222.0.32 () {56 vars in 1183 bytes} [Mon May 30 11:52:34 2022] GET /identity/v3/domains/cfedc9def0e84875b9915ef0717522d3 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:34.486958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ea1d4-3d61-4b66-840f-d62e26e76dce None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:34.586098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ea1d4-3d61-4b66-840f-d62e26e76dce None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:34.620371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13c7ae-fb1c-4260-9ea9-347680b54d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:34.621046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13c7ae-fb1c-4260-9ea9-347680b54d4b None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:34.621333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13c7ae-fb1c-4260-9ea9-347680b54d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:34.621613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13c7ae-fb1c-4260-9ea9-347680b54d4b None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:34.622332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13c7ae-fb1c-4260-9ea9-347680b54d4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:34.623765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13c7ae-fb1c-4260-9ea9-347680b54d4b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:34.626991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:34.653625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 584/1173] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:52:34 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:52:36.972541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6dc7b-9d1c-4daa-903f-480ccf66bd4f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:36.973041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6dc7b-9d1c-4daa-903f-480ccf66bd4f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:36.973041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6dc7b-9d1c-4daa-903f-480ccf66bd4f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:36.974381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 590/1174] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:36.979844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c573fb9-e173-4478-af3a-47bfb8aacf28 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:36.980274 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c573fb9-e173-4478-af3a-47bfb8aacf28 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:36.980478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c573fb9-e173-4478-af3a-47bfb8aacf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:36.997703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9c573fb9-e173-4478-af3a-47bfb8aacf28 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:36.998891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9c573fb9-e173-4478-af3a-47bfb8aacf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:37.065970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 585/1175] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:37.090898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6dc7b-9d1c-4daa-903f-480ccf66bd4f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:37.191233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6dc7b-9d1c-4daa-903f-480ccf66bd4f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:37.225032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec73f5-0d1e-4980-8d67-8453761e7db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:37.225830 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec73f5-0d1e-4980-8d67-8453761e7db0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:37.226119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec73f5-0d1e-4980-8d67-8453761e7db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:37.226403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec73f5-0d1e-4980-8d67-8453761e7db0 None admin] PATH_INFO: `/v3/domains/cfedc9def0e84875b9915ef0717522d3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:37.227599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec73f5-0d1e-4980-8d67-8453761e7db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cfedc9def0e84875b9915ef0717522d3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:37.229080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec73f5-0d1e-4980-8d67-8453761e7db0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:37.230698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 591/1176] 10.222.0.32 () {56 vars in 1183 bytes} [Mon May 30 11:52:37 2022] GET /identity/v3/domains/cfedc9def0e84875b9915ef0717522d3 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:37.236580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c573fb9-e173-4478-af3a-47bfb8aacf28 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:37.334488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c573fb9-e173-4478-af3a-47bfb8aacf28 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:37.351203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:37.369649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03628d7-767b-4665-adf4-530c35c8fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:37.370321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03628d7-767b-4665-adf4-530c35c8fab1 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:37.370608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03628d7-767b-4665-adf4-530c35c8fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:37.370887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03628d7-767b-4665-adf4-530c35c8fab1 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:37.371598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03628d7-767b-4665-adf4-530c35c8fab1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:37.373061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03628d7-767b-4665-adf4-530c35c8fab1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:37.392839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 586/1177] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:52:37 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:52:39.721158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e11062-c0a7-441a-9246-e08c9e696d3b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:39.722456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e11062-c0a7-441a-9246-e08c9e696d3b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:39.722456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e11062-c0a7-441a-9246-e08c9e696d3b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:39.723825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 592/1178] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:39.728916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb382f-c585-49ca-b105-276b540afffd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:39.729354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb382f-c585-49ca-b105-276b540afffd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:39.729473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb382f-c585-49ca-b105-276b540afffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:39.747299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a2cb382f-c585-49ca-b105-276b540afffd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:39.748503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a2cb382f-c585-49ca-b105-276b540afffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:39.826898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 587/1179] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:39.851317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e11062-c0a7-441a-9246-e08c9e696d3b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:39.908595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:39.953298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e11062-c0a7-441a-9246-e08c9e696d3b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:39.986759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604acf3a-e139-4963-840d-76fe550805e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:39.988101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604acf3a-e139-4963-840d-76fe550805e7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:39.988413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604acf3a-e139-4963-840d-76fe550805e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:39.988695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604acf3a-e139-4963-840d-76fe550805e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:39.998765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604acf3a-e139-4963-840d-76fe550805e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-604acf3a-e139-4963-840d-76fe550805e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:52:40.006503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:52:40.007507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:52:40.008400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 593/1180] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:52:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:52:40.013781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb382f-c585-49ca-b105-276b540afffd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:40.114518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb382f-c585-49ca-b105-276b540afffd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:40.116027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:40.152780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:40.153653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:40.153801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:40.154038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:40.154808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:40.155118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:40.156545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:40.166342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 588/1181] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:52:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:40.172142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604acf3a-e139-4963-840d-76fe550805e7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:40.271407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604acf3a-e139-4963-840d-76fe550805e7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:40.285134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:40.306038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c4043-cf1e-4770-a985-0abd562a037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:40.306913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c4043-cf1e-4770-a985-0abd562a037c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:40.307229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c4043-cf1e-4770-a985-0abd562a037c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:40.307513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c4043-cf1e-4770-a985-0abd562a037c None admin] PATH_INFO: `/v3/users/d5d3426c3b9e41d281bc98739160c191` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:40.321914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c4043-cf1e-4770-a985-0abd562a037c None admin] RBAC: Authorizing `identity:get_user(user_id=d5d3426c3b9e41d281bc98739160c191)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:40.323403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c4043-cf1e-4770-a985-0abd562a037c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:40.325080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 594/1182] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:40 2022] GET /identity/v3/users/d5d3426c3b9e41d281bc98739160c191 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:40.331247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:40.431230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0bdbd-5200-4c9c-ba48-6ebdedd1e8db None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:40.464412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d45f9-f3f9-4327-b726-2f85eb15a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:40.465054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d45f9-f3f9-4327-b726-2f85eb15a8cf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:40.465399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d45f9-f3f9-4327-b726-2f85eb15a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:40.465619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d45f9-f3f9-4327-b726-2f85eb15a8cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:40.496613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:40.533415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d45f9-f3f9-4327-b726-2f85eb15a8cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:40.535225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d45f9-f3f9-4327-b726-2f85eb15a8cf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:40.632215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 589/1183] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:52:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:52:40.638087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c4043-cf1e-4770-a985-0abd562a037c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:40.736749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c4043-cf1e-4770-a985-0abd562a037c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:40.771132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29413efb-09d0-4785-a7a5-a9ccd3a7d52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:40.771992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29413efb-09d0-4785-a7a5-a9ccd3a7d52f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:40.772311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29413efb-09d0-4785-a7a5-a9ccd3a7d52f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:40.772595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29413efb-09d0-4785-a7a5-a9ccd3a7d52f None admin] PATH_INFO: `/v3/projects/15fbfb4a9d1043f9b6e50b53fc946a96` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:40.773784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29413efb-09d0-4785-a7a5-a9ccd3a7d52f None admin] RBAC: Authorizing `identity:get_project(project_id=15fbfb4a9d1043f9b6e50b53fc946a96)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:40.775222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29413efb-09d0-4785-a7a5-a9ccd3a7d52f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:40.776959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 595/1184] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:40 2022] GET /identity/v3/projects/15fbfb4a9d1043f9b6e50b53fc946a96 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:40.783059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d45f9-f3f9-4327-b726-2f85eb15a8cf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:40.845475 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:40.883675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d45f9-f3f9-4327-b726-2f85eb15a8cf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:40.919385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b996c-7ddc-4929-aba3-b0cf9419fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:40.920029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b996c-7ddc-4929-aba3-b0cf9419fe9e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:40.920349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b996c-7ddc-4929-aba3-b0cf9419fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:40.920628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b996c-7ddc-4929-aba3-b0cf9419fe9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:40.921454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b996c-7ddc-4929-aba3-b0cf9419fe9e None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=15fbfb4a9d1043f9b6e50b53fc946a96, user.id=d5d3426c3b9e41d281bc98739160c191) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:40.921765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b996c-7ddc-4929-aba3-b0cf9419fe9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:40.923204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b996c-7ddc-4929-aba3-b0cf9419fe9e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:40.929311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 590/1185] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:52:40 2022] GET /identity/v3/role_assignments?user.id=d5d3426c3b9e41d281bc98739160c191&scope.project.id=15fbfb4a9d1043f9b6e50b53fc946a96&role.id=4d21498572b14478a996ac538be5382e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:43.182629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c4d39-5c78-494f-af4c-6bf4eb607746 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:43.183178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c4d39-5c78-494f-af4c-6bf4eb607746 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:43.183178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c4d39-5c78-494f-af4c-6bf4eb607746 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:43.184601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 596/1186] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:43.189744 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcee43-7b0f-4328-b713-5b6eb7d24be9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:43.190350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcee43-7b0f-4328-b713-5b6eb7d24be9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:43.190638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcee43-7b0f-4328-b713-5b6eb7d24be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:43.211275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5cdcee43-7b0f-4328-b713-5b6eb7d24be9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:43.212491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5cdcee43-7b0f-4328-b713-5b6eb7d24be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:43.279722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 591/1187] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:43.303289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c4d39-5c78-494f-af4c-6bf4eb607746 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:43.346697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:43.405764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c4d39-5c78-494f-af4c-6bf4eb607746 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:43.438372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba119-0b50-4294-bd8b-f2538a37c913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:43.439095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba119-0b50-4294-bd8b-f2538a37c913 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:43.439384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba119-0b50-4294-bd8b-f2538a37c913 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:43.439665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba119-0b50-4294-bd8b-f2538a37c913 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:43.450294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ba119-0b50-4294-bd8b-f2538a37c913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-615ba119-0b50-4294-bd8b-f2538a37c913 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:52:43.458406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:52:43.459501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:52:43.460589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 597/1188] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:52:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:52:43.465699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcee43-7b0f-4328-b713-5b6eb7d24be9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:43.522225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:43.563609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcee43-7b0f-4328-b713-5b6eb7d24be9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:43.598193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:43.598871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:43.599158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:43.599435 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:43.600217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:43.600534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:43.601896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:43.609957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 592/1189] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:52:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:43.616129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba119-0b50-4294-bd8b-f2538a37c913 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:43.713183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:43.716351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba119-0b50-4294-bd8b-f2538a37c913 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:43.749951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3eec1b-c658-4bbd-9124-ba64fa2dc94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:43.750813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eec1b-c658-4bbd-9124-ba64fa2dc94b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:43.751097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eec1b-c658-4bbd-9124-ba64fa2dc94b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:43.751371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3eec1b-c658-4bbd-9124-ba64fa2dc94b None admin] PATH_INFO: `/v3/users/d5d3426c3b9e41d281bc98739160c191` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:43.753366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3eec1b-c658-4bbd-9124-ba64fa2dc94b None admin] RBAC: Authorizing `identity:get_user(user_id=d5d3426c3b9e41d281bc98739160c191)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:43.754914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3eec1b-c658-4bbd-9124-ba64fa2dc94b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:43.756561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 598/1190] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:43 2022] GET /identity/v3/users/d5d3426c3b9e41d281bc98739160c191 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:43.762334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:43.861321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0f8a1-12ab-4115-a3f8-b8c1654dade9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:43.882665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:43.899105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62b2c5-3e4e-4111-b3f6-6cfd776d7f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:43.899964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62b2c5-3e4e-4111-b3f6-6cfd776d7f0c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:43.900280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62b2c5-3e4e-4111-b3f6-6cfd776d7f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:43.900559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62b2c5-3e4e-4111-b3f6-6cfd776d7f0c None admin] PATH_INFO: `/v3/projects/15fbfb4a9d1043f9b6e50b53fc946a96` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:43.901728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62b2c5-3e4e-4111-b3f6-6cfd776d7f0c None admin] RBAC: Authorizing `identity:get_project(project_id=15fbfb4a9d1043f9b6e50b53fc946a96)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:43.903150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62b2c5-3e4e-4111-b3f6-6cfd776d7f0c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:43.904853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 593/1191] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:43 2022] GET /identity/v3/projects/15fbfb4a9d1043f9b6e50b53fc946a96 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:43.911384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3eec1b-c658-4bbd-9124-ba64fa2dc94b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:44.012235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3eec1b-c658-4bbd-9124-ba64fa2dc94b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:44.046161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e7d1d-8acb-4e53-9979-183c469aee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:44.047052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e7d1d-8acb-4e53-9979-183c469aee2c None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:44.047330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e7d1d-8acb-4e53-9979-183c469aee2c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:44.047601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e7d1d-8acb-4e53-9979-183c469aee2c None admin] PATH_INFO: `/v3/projects/15fbfb4a9d1043f9b6e50b53fc946a96/users/d5d3426c3b9e41d281bc98739160c191/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:44.050324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e7d1d-8acb-4e53-9979-183c469aee2c None admin] RBAC: Authorizing `identity:create_grant(project_id=15fbfb4a9d1043f9b6e50b53fc946a96, user_id=d5d3426c3b9e41d281bc98739160c191, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:44.051910 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e7d1d-8acb-4e53-9979-183c469aee2c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:44.066254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 599/1192] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:52:43 2022] PUT /identity/v3/projects/15fbfb4a9d1043f9b6e50b53fc946a96/users/d5d3426c3b9e41d281bc98739160c191/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:52:46.348465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bf174-7087-48e9-93fe-bd46609ee05b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:46.349000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bf174-7087-48e9-93fe-bd46609ee05b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:46.349000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bf174-7087-48e9-93fe-bd46609ee05b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:46.350387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 594/1193] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:52:46.355611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9073b-eddf-480b-83e2-97622fd8882d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:46.356044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9073b-eddf-480b-83e2-97622fd8882d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:46.356184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9073b-eddf-480b-83e2-97622fd8882d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:46.373401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-78a9073b-eddf-480b-83e2-97622fd8882d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:52:46.374598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-78a9073b-eddf-480b-83e2-97622fd8882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:52:46.399337 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:46.443847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 600/1194] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:52:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:52:46.468530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bf174-7087-48e9-93fe-bd46609ee05b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:46.567912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bf174-7087-48e9-93fe-bd46609ee05b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:46.601787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af61e9-5474-47f7-ab5c-5759a444b076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:46.602529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af61e9-5474-47f7-ab5c-5759a444b076 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:46.602813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af61e9-5474-47f7-ab5c-5759a444b076 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:46.603094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af61e9-5474-47f7-ab5c-5759a444b076 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:46.614194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af61e9-5474-47f7-ab5c-5759a444b076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b6af61e9-5474-47f7-ab5c-5759a444b076 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:52:46.621832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:52:46.622942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:52:46.623846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 595/1195] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:52:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:52:46.629047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9073b-eddf-480b-83e2-97622fd8882d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:46.725408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9073b-eddf-480b-83e2-97622fd8882d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:46.761025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:46.761726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:46.762015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:46.762295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:46.763047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:46.763359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:46.764788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:46.773772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 601/1196] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:52:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:46.779501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af61e9-5474-47f7-ab5c-5759a444b076 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:46.827992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:46.878308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af61e9-5474-47f7-ab5c-5759a444b076 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:46.913572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f9790-8160-4951-b3e8-43462c29db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:46.914438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9790-8160-4951-b3e8-43462c29db22 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:46.914720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9790-8160-4951-b3e8-43462c29db22 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:46.914993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9790-8160-4951-b3e8-43462c29db22 None admin] PATH_INFO: `/v3/users/d5d3426c3b9e41d281bc98739160c191` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:46.916150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f9790-8160-4951-b3e8-43462c29db22 None admin] RBAC: Authorizing `identity:get_user(user_id=d5d3426c3b9e41d281bc98739160c191)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:46.917584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f9790-8160-4951-b3e8-43462c29db22 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:46.919178 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 596/1197] 10.222.0.32 () {56 vars in 1177 bytes} [Mon May 30 11:52:46 2022] GET /identity/v3/users/d5d3426c3b9e41d281bc98739160c191 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:46.925096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:46.946914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:47.024341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a8eac-37f0-48d8-bd2e-480021e07c8d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:47.057490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f2590-11d6-4fb7-a5d2-5d2620fad263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:47.058135 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f2590-11d6-4fb7-a5d2-5d2620fad263 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:47.058419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f2590-11d6-4fb7-a5d2-5d2620fad263 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:47.058695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f2590-11d6-4fb7-a5d2-5d2620fad263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:47.124063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f2590-11d6-4fb7-a5d2-5d2620fad263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:47.125474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f2590-11d6-4fb7-a5d2-5d2620fad263 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:47.148760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:47.230070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 602/1198] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:52:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 30 11:52:47.235946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f9790-8160-4951-b3e8-43462c29db22 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:47.333770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f9790-8160-4951-b3e8-43462c29db22 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:47.350342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:47.368890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3afd46-746a-400f-963d-899c0a9ab23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:47.370361 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3afd46-746a-400f-963d-899c0a9ab23b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:47.370882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3afd46-746a-400f-963d-899c0a9ab23b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:47.371402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3afd46-746a-400f-963d-899c0a9ab23b None admin] PATH_INFO: `/v3/projects/15fbfb4a9d1043f9b6e50b53fc946a96` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:47.373126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3afd46-746a-400f-963d-899c0a9ab23b None admin] RBAC: Authorizing `identity:get_project(project_id=15fbfb4a9d1043f9b6e50b53fc946a96)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:47.374554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3afd46-746a-400f-963d-899c0a9ab23b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:47.376259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 597/1199] 10.222.0.32 () {56 vars in 1186 bytes} [Mon May 30 11:52:47 2022] GET /identity/v3/projects/15fbfb4a9d1043f9b6e50b53fc946a96 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:47.383079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f2590-11d6-4fb7-a5d2-5d2620fad263 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:52:47.491212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f2590-11d6-4fb7-a5d2-5d2620fad263 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:52:47.512179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:47.527729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04552-61cb-4cc8-9f6c-781b05b93afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:52:47.528425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04552-61cb-4cc8-9f6c-781b05b93afc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:52:47.528712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04552-61cb-4cc8-9f6c-781b05b93afc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:52:47.528983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04552-61cb-4cc8-9f6c-781b05b93afc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:52:47.529820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04552-61cb-4cc8-9f6c-781b05b93afc None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=15fbfb4a9d1043f9b6e50b53fc946a96, user.id=d5d3426c3b9e41d281bc98739160c191) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:52:47.530134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04552-61cb-4cc8-9f6c-781b05b93afc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:52:47.531569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04552-61cb-4cc8-9f6c-781b05b93afc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:52:47.537962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 603/1200] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:52:47 2022] GET /identity/v3/role_assignments?user.id=d5d3426c3b9e41d281bc98739160c191&scope.project.id=15fbfb4a9d1043f9b6e50b53fc946a96&role.id=4d21498572b14478a996ac538be5382e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:52:57.363784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:52:57.535370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=8 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:50.037372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1a0a4-fe8b-4133-875e-b2560e42a8af None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:50.037372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1a0a4-fe8b-4133-875e-b2560e42a8af None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:50.037372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1a0a4-fe8b-4133-875e-b2560e42a8af None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:50.037372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 598/1201] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:53:50.044705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e130-fc02-42c6-b9e9-36e38b79294d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:50.045404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e130-fc02-42c6-b9e9-36e38b79294d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:50.045404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e130-fc02-42c6-b9e9-36e38b79294d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:50.071079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-18f0e130-fc02-42c6-b9e9-36e38b79294d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:53:50.072836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-18f0e130-fc02-42c6-b9e9-36e38b79294d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:53:50.184377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 604/1202] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:53:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:53:50.206785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1a0a4-fe8b-4133-875e-b2560e42a8af None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:50.353543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1a0a4-fe8b-4133-875e-b2560e42a8af None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:50.399395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:50.400551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:50.400757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:50.401127 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:50.415060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:50.416957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:53:50.440146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:50.441235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:50.442258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:53:50.442258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:53:50.442258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:53:50.442258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:53:50.442258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 599/1203] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:50.448819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0e130-fc02-42c6-b9e9-36e38b79294d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:50.583150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0e130-fc02-42c6-b9e9-36e38b79294d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:50.630517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:50.637183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39632b-2e1c-48e8-88cb-a566aba7e8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:50.637885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39632b-2e1c-48e8-88cb-a566aba7e8dd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:50.638186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39632b-2e1c-48e8-88cb-a566aba7e8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:50.638453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39632b-2e1c-48e8-88cb-a566aba7e8dd None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:50.639183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39632b-2e1c-48e8-88cb-a566aba7e8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:50.639517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39632b-2e1c-48e8-88cb-a566aba7e8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:50.643792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39632b-2e1c-48e8-88cb-a566aba7e8dd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:50.657463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 605/1204] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:53:50.663094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:50.727852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:50.790887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c519fc-be34-46d1-bbb5-10d2fc951596 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:50.834002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dd073-3d0b-4007-9e0c-49326c2bfb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:50.834981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dd073-3d0b-4007-9e0c-49326c2bfb0b None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:50.835439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dd073-3d0b-4007-9e0c-49326c2bfb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:50.835958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dd073-3d0b-4007-9e0c-49326c2bfb0b None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:50.836716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dd073-3d0b-4007-9e0c-49326c2bfb0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:50.838158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dd073-3d0b-4007-9e0c-49326c2bfb0b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:50.872204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 600/1205] 10.222.0.32 () {60 vars in 1132 bytes} [Mon May 30 11:53:50 2022] POST /identity/v3/users => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 11:53:53.597107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a074342-ead7-40ab-8af3-f93c49b12748 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:53.597665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a074342-ead7-40ab-8af3-f93c49b12748 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:53.598063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a074342-ead7-40ab-8af3-f93c49b12748 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:53.599734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 606/1206] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:53:53.605537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5c579-049b-40d3-9f5f-aab9fb9e2b7a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:53.606021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5c579-049b-40d3-9f5f-aab9fb9e2b7a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:53.606132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5c579-049b-40d3-9f5f-aab9fb9e2b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:53.623856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67b5c579-049b-40d3-9f5f-aab9fb9e2b7a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:53:53.625731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67b5c579-049b-40d3-9f5f-aab9fb9e2b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:53:53.715728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 601/1207] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:53:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:53:53.745436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a074342-ead7-40ab-8af3-f93c49b12748 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:53.871995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a074342-ead7-40ab-8af3-f93c49b12748 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:53.913284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4724660-a40f-4213-9c53-5c1c8931fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:53.914461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4724660-a40f-4213-9c53-5c1c8931fd89 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:53.914801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4724660-a40f-4213-9c53-5c1c8931fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:53.915243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4724660-a40f-4213-9c53-5c1c8931fd89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:53.929375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4724660-a40f-4213-9c53-5c1c8931fd89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f4724660-a40f-4213-9c53-5c1c8931fd89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:53:53.938246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:53.939718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:53:53.940699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 607/1208] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:53:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:53.946407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5c579-049b-40d3-9f5f-aab9fb9e2b7a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:53.991670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:54.064021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5c579-049b-40d3-9f5f-aab9fb9e2b7a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:54.100990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:54.101659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:54.101923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:54.102228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:54.102969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:54.103342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:54.104700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:54.114263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 602/1209] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:53:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:53:54.120018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4724660-a40f-4213-9c53-5c1c8931fd89 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:54.189611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:54.252475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4724660-a40f-4213-9c53-5c1c8931fd89 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:54.295525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:54.297450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:54.297811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:54.298343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:54.312150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:54.314215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:53:54.335095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:54.336310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:54.342004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:53:54.342004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:53:54.342004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:53:54.342004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:53:54.342004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 608/1210] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:54.342462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:54.459492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:54.462786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a49117-0bc2-4ac4-9526-8a666be5db59 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:54.499755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:54.500472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:54.500746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:54.501023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:54.501749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:54.502075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:54.503589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:54.517389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 603/1211] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:53:54.522858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:54.643745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b822f0e-5cbe-4925-becc-0a182fa0d19b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:54.644867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:54.684516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:54.685332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:54.685678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:54.688269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:54.695583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:54.697297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:53:54.714845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:54.716014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:53:54.717264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 609/1212] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:53:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:54.724474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:54.838349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2fbdf-ccc1-41ee-b3a5-e2008233dc79 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:54.878474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:54.879198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:54.879482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:54.879762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:54.880652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:54.881652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:54.882406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:54.902406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 604/1213] 10.222.0.32 () {56 vars in 1143 bytes} [Mon May 30 11:53:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:53:54.908502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:55.031736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97819f-bb69-40ff-a318-f908475f6a78 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:55.072639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b9c4c-9c96-431b-a761-87924da92244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:55.073603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b9c4c-9c96-431b-a761-87924da92244 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:55.073965 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b9c4c-9c96-431b-a761-87924da92244 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:55.074289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b9c4c-9c96-431b-a761-87924da92244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:55.116710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:55.164824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b9c4c-9c96-431b-a761-87924da92244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:55.166581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b9c4c-9c96-431b-a761-87924da92244 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:55.299947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 610/1214] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:53:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:53:55.306185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:55.327932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:55.423047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1971859-2fe5-43e1-8f47-391e5ea8b6cb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:55.460403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:55.461392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:55.461831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:55.462216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:55.476491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:55.478258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:53:55.498298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:55.499621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:55.500766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:53:55.500766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:53:55.500766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:53:55.500766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:53:55.501848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 605/1215] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:55.507252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b9c4c-9c96-431b-a761-87924da92244 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:55.582098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:55.632060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b9c4c-9c96-431b-a761-87924da92244 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:55.670537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:55.671959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:55.672445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:55.672881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:55.673746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:55.674231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:55.675623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:55.692202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 611/1216] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:53:55.695468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:55.794556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:55.835050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbf7bd-16a6-4023-91e7-d9b24bbdc73f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:55.873088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:55.874176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:55.874479 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:55.874842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:55.884189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:55.885825 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:53:55.903221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:55.904418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:55.905398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:53:55.905398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:53:55.905398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:53:55.905398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:53:55.905398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:53:55.905398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:53:55.905398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:53:55.905398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:53:55.911504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 606/1217] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:53:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:55.911663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:56.032841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcb3f-cbc5-4aae-993e-128379bb49aa None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:56.058949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:56.080083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e47a9b-f167-4736-8386-ea32fc004db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:56.081056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e47a9b-f167-4736-8386-ea32fc004db1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:56.081332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e47a9b-f167-4736-8386-ea32fc004db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:56.081711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e47a9b-f167-4736-8386-ea32fc004db1 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:56.082550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e47a9b-f167-4736-8386-ea32fc004db1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:56.082992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e47a9b-f167-4736-8386-ea32fc004db1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:56.084859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e47a9b-f167-4736-8386-ea32fc004db1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:56.106823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 612/1218] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:53:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:53:56.108915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:56.265297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6de649-766e-484c-8427-6f73c4e3ad5d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:56.307902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:56.315724 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2237cb-2f1d-486f-94bb-8647b503a6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:56.317007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2237cb-2f1d-486f-94bb-8647b503a6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:56.317745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2237cb-2f1d-486f-94bb-8647b503a6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:56.318408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2237cb-2f1d-486f-94bb-8647b503a6c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:56.319320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2237cb-2f1d-486f-94bb-8647b503a6c9 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=6c654dad1f844fe9995c594116619216) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:56.319661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2237cb-2f1d-486f-94bb-8647b503a6c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:56.321229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2237cb-2f1d-486f-94bb-8647b503a6c9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:56.336192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 607/1219] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:53:56 2022] GET /identity/v3/role_assignments?user.id=6c654dad1f844fe9995c594116619216&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:53:58.697484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb08bd-36ec-46ff-a0ac-49d2e7053c54 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:58.698015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb08bd-36ec-46ff-a0ac-49d2e7053c54 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:58.698104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb08bd-36ec-46ff-a0ac-49d2e7053c54 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:58.699424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 613/1220] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:53:58.706635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb072a3-381a-47aa-979e-4107c5c988bc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:58.707156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb072a3-381a-47aa-979e-4107c5c988bc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:58.707237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb072a3-381a-47aa-979e-4107c5c988bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:58.727669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-eeb072a3-381a-47aa-979e-4107c5c988bc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:53:58.729302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eeb072a3-381a-47aa-979e-4107c5c988bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:53:58.812121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 608/1221] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:53:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:53:58.840468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb08bd-36ec-46ff-a0ac-49d2e7053c54 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:58.944273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb08bd-36ec-46ff-a0ac-49d2e7053c54 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:58.979740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42819ab3-19de-4273-bce1-e20315138b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:58.980588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42819ab3-19de-4273-bce1-e20315138b55 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:58.980880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42819ab3-19de-4273-bce1-e20315138b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:58.981162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42819ab3-19de-4273-bce1-e20315138b55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:58.992658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42819ab3-19de-4273-bce1-e20315138b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-42819ab3-19de-4273-bce1-e20315138b55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:53:59.000787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:59.007622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:53:59.008504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 614/1222] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:53:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:59.009211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb072a3-381a-47aa-979e-4107c5c988bc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:59.117942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb072a3-381a-47aa-979e-4107c5c988bc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:59.145571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:59.157541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:59.158214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:59.158526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:59.158776 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:59.159531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:59.159845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:59.161280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:59.169545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 609/1223] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:53:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:53:59.177038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42819ab3-19de-4273-bce1-e20315138b55 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:59.226635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:59.290804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42819ab3-19de-4273-bce1-e20315138b55 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:59.334857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:59.335663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:59.335950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:59.336263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:59.348538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:59.350146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:53:59.369713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:59.371134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:59.372520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:53:59.372520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:53:59.372520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:53:59.372520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:53:59.372520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 615/1224] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:59.377678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:59.491368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b26a8-8146-40d8-8c07-f55f7247d6a5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:59.531253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:59.531962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:59.532280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:59.532566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:59.533331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:59.533673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:59.535231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:59.548717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 610/1225] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:53:59.554365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:59.659603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:59.666747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca6e9d-e982-429e-ab70-baa1d989a84b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:59.704743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:59.705593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:59.705928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:59.706254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:59.714677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:59.716320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:53:59.732334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:53:59.734170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:53:59.735358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 616/1226] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:53:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:53:59.740082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:53:59.791587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:53:59.852289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ad906-2a48-413f-be69-a722a0714a78 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:53:59.895311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:53:59.896131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:53:59.896447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:53:59.896730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:53:59.897648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:53:59.898232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:53:59.900654 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:53:59.920081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 611/1227] 10.222.0.32 () {56 vars in 1143 bytes} [Mon May 30 11:53:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:53:59.926509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:00.036449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e56b7-0aeb-439d-94b7-9bb11508609d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:00.073104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:00.073843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:00.074134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:00.074418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:00.088460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:00.090004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:00.112130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:00.113152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:00.114081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:00.114081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:00.114081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:00.114081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:00.114081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 617/1228] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:54:00.119204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:00.244100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acced8-77a7-4453-9524-971c7d3199b1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:00.246712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:00.285875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:00.286548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:00.286827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:00.287245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:00.287829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:00.288150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:00.289884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:00.307363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 612/1229] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:00.312992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:00.324992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:00.429525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe72b9-3418-472f-abe6-ad5272c01e4f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:00.466786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:00.468289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:00.468583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:00.468866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:00.478730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:00.480366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:54:00.496072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:00.497100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:00.498031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 618/1230] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:00.503781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:00.620144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84450675-5f05-4d82-a6be-40d84c711564 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:00.662602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1495e-6111-48fc-9f53-8077af467521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:00.663398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1495e-6111-48fc-9f53-8077af467521 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:00.663683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1495e-6111-48fc-9f53-8077af467521 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:00.663961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1495e-6111-48fc-9f53-8077af467521 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:00.664805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1495e-6111-48fc-9f53-8077af467521 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:00.665127 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1495e-6111-48fc-9f53-8077af467521 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:00.666679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1495e-6111-48fc-9f53-8077af467521 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:00.681864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 613/1231] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:54:00.690031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:00.766089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:00.806645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbdcef-fa5f-418f-b867-c20fead888e5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:00.845610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8a316-a340-4311-a569-1311c267b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:00.846833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a316-a340-4311-a569-1311c267b8fc None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:00.847027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a316-a340-4311-a569-1311c267b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:00.847393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8a316-a340-4311-a569-1311c267b8fc None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/6c654dad1f844fe9995c594116619216/roles/126791116ac8403986eead13c8c6dd07` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:00.863235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8a316-a340-4311-a569-1311c267b8fc None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=6c654dad1f844fe9995c594116619216, role_id=126791116ac8403986eead13c8c6dd07)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:00.864896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8a316-a340-4311-a569-1311c267b8fc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:00.878089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 619/1232] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:54:00 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/6c654dad1f844fe9995c594116619216/roles/126791116ac8403986eead13c8c6dd07 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:54:03.370468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b65e-1135-4048-94a4-8896fa67a71e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:03.371025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b65e-1135-4048-94a4-8896fa67a71e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:03.371025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4b65e-1135-4048-94a4-8896fa67a71e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:03.372513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 614/1233] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:54:03.378322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a3569-186b-46f5-8d94-96cc9cbbe307 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:03.378814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a3569-186b-46f5-8d94-96cc9cbbe307 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:03.378908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a3569-186b-46f5-8d94-96cc9cbbe307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:03.398436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f88a3569-186b-46f5-8d94-96cc9cbbe307 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:54:03.399659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f88a3569-186b-46f5-8d94-96cc9cbbe307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:54:03.476636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 620/1234] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:54:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:54:03.506803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b65e-1135-4048-94a4-8896fa67a71e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:03.540209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:03.630822 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4b65e-1135-4048-94a4-8896fa67a71e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:03.671813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190552f-82eb-4f66-9539-175e740e51ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:03.672653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190552f-82eb-4f66-9539-175e740e51ae None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:03.672935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190552f-82eb-4f66-9539-175e740e51ae None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:03.673208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190552f-82eb-4f66-9539-175e740e51ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:03.685342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190552f-82eb-4f66-9539-175e740e51ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6190552f-82eb-4f66-9539-175e740e51ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:54:03.694109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:03.696098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:03.698101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 615/1235] 10.222.0.32 () {56 vars in 1102 bytes} [Mon May 30 11:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:54:03.701580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a3569-186b-46f5-8d94-96cc9cbbe307 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:03.758134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:03.821597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a3569-186b-46f5-8d94-96cc9cbbe307 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:03.865480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:03.866236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:03.866558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:03.866844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:03.867643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:03.867957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:03.869482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:03.878606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 621/1236] 10.222.0.32 () {56 vars in 1103 bytes} [Mon May 30 11:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:54:03.884583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190552f-82eb-4f66-9539-175e740e51ae None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:03.981932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:04.002113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190552f-82eb-4f66-9539-175e740e51ae None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:04.041288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:04.042211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:04.042519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:04.042895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:04.056352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:04.057815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:04.075922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:04.077758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:04.078687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:04.078687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:04.078687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:04.078687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:04.078687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 616/1237] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:54:04.083605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:04.197468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c9ec9-076a-4f5a-a43b-47474b7d18ad None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:04.208124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:04.236587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:04.237266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:04.237556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:04.237842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:04.238660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:04.238987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:04.240533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:04.253879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 622/1238] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:04.259868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:04.381796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c6b1d-904b-41e0-9b97-d080022168ac None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:04.421811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:04.422438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:04.422721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:04.423013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:04.431378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:04.431913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:04.433786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:54:04.449924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:04.451119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:04.452914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 617/1239] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:54:04.457710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:04.573530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f5dc4-1107-45cb-8a62-f4320ef65093 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:04.611230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:04.611883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:04.612295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:04.612474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:04.613325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:04.613672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:04.615074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:04.632813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 623/1240] 10.222.0.32 () {56 vars in 1143 bytes} [Mon May 30 11:54:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:04.639598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:04.793710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca9dab-a0a4-4763-ad11-a29c634bd4dd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:04.834116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fd247-9ba8-42a7-9419-10b8ee4336f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:04.834890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fd247-9ba8-42a7-9419-10b8ee4336f1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:04.835280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fd247-9ba8-42a7-9419-10b8ee4336f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:04.835663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fd247-9ba8-42a7-9419-10b8ee4336f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:04.921618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:04.929720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fd247-9ba8-42a7-9419-10b8ee4336f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:04.931523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fd247-9ba8-42a7-9419-10b8ee4336f1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:05.046275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 618/1241] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:54:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:54:05.051757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:05.080289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:05.160935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ac500-e462-4d60-9888-ab01a5f0503e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:05.201113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:05.202013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:05.202279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:05.202692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:05.214276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:05.215753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:05.237345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:05.238522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:05.239778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:05.239778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:05.239778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:05.239778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:05.239778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 624/1242] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:05.244245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fd247-9ba8-42a7-9419-10b8ee4336f1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:05.356348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:05.365683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fd247-9ba8-42a7-9419-10b8ee4336f1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:05.406996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:05.407684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:05.407938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:05.408244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:05.408975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:05.409290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:05.410778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:05.426013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 619/1243] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:05.432330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:05.534946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:05.559122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6aaa09-23a5-43a5-b829-11b76e314285 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:05.598190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:05.599108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:05.599397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:05.599681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:05.607647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:05.609308 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:54:05.623646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:05.625220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:05.626819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 625/1244] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:05.630871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:05.749319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdd845-da29-428e-9919-b0509592f0d8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:05.787945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f941d9b-2db7-418c-9ff5-8da3856ea081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:05.788732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f941d9b-2db7-418c-9ff5-8da3856ea081 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:05.789019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f941d9b-2db7-418c-9ff5-8da3856ea081 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:05.789302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f941d9b-2db7-418c-9ff5-8da3856ea081 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:05.790176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f941d9b-2db7-418c-9ff5-8da3856ea081 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:05.790490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f941d9b-2db7-418c-9ff5-8da3856ea081 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:05.792129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f941d9b-2db7-418c-9ff5-8da3856ea081 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:05.807278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 620/1245] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:05.814179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:05.933028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe25e8-cf27-472f-834f-2d7575a27e68 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:05.971902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad644c1c-1a58-4d3d-aa76-34b0985034e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:05.972684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad644c1c-1a58-4d3d-aa76-34b0985034e8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:05.972997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad644c1c-1a58-4d3d-aa76-34b0985034e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:05.973300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad644c1c-1a58-4d3d-aa76-34b0985034e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:05.974195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad644c1c-1a58-4d3d-aa76-34b0985034e8 None admin] RBAC: Adding query filter params (role.id=126791116ac8403986eead13c8c6dd07, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=6c654dad1f844fe9995c594116619216) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:05.974513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad644c1c-1a58-4d3d-aa76-34b0985034e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:05.976071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad644c1c-1a58-4d3d-aa76-34b0985034e8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:05.982635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 626/1246] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:54:05 2022] GET /identity/v3/role_assignments?user.id=6c654dad1f844fe9995c594116619216&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=126791116ac8403986eead13c8c6dd07 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:08.335225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801b578-c7c5-422a-a9d9-f370efa7e6bb None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:08.335883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801b578-c7c5-422a-a9d9-f370efa7e6bb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:08.335883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801b578-c7c5-422a-a9d9-f370efa7e6bb None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:08.337329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 621/1247] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:54:08.344619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3b0e2-5845-4a56-8003-50b6d51f60d8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:08.345213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3b0e2-5845-4a56-8003-50b6d51f60d8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:08.345213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3b0e2-5845-4a56-8003-50b6d51f60d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:08.350431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:08.365792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9ae3b0e2-5845-4a56-8003-50b6d51f60d8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:54:08.367287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ae3b0e2-5845-4a56-8003-50b6d51f60d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:54:08.447917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 627/1248] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:54:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:54:08.474867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801b578-c7c5-422a-a9d9-f370efa7e6bb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:08.497169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:08.595386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801b578-c7c5-422a-a9d9-f370efa7e6bb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:08.633553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32bcdb-8bec-42a5-bf65-09ad3ab5dacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:08.634454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32bcdb-8bec-42a5-bf65-09ad3ab5dacc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:08.634815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32bcdb-8bec-42a5-bf65-09ad3ab5dacc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:08.635101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32bcdb-8bec-42a5-bf65-09ad3ab5dacc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:08.648104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32bcdb-8bec-42a5-bf65-09ad3ab5dacc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ba32bcdb-8bec-42a5-bf65-09ad3ab5dacc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:54:08.657324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:08.660374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:08.661512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 622/1249] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:54:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:08.664732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3b0e2-5845-4a56-8003-50b6d51f60d8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:08.778869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3b0e2-5845-4a56-8003-50b6d51f60d8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:08.814051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:08.820107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:08.820806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:08.821095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:08.821371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:08.822126 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:08.822439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:08.823867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:08.832626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 628/1250] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:54:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:08.838256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32bcdb-8bec-42a5-bf65-09ad3ab5dacc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:08.928752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:08.956785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32bcdb-8bec-42a5-bf65-09ad3ab5dacc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:08.996985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:08.997808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:08.998097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:08.998442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:09.009939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:09.011406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:09.031222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:09.037240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:09.038319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:09.038319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:09.038319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:09.038319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:09.038319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 623/1251] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 30 11:54:09.039455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:09.150642 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68a29b-b3b5-45b3-9f9b-e3d2e0cd190c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:09.187984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:09.188771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:09.189084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:09.189384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:09.190366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:09.190582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:09.192132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:09.211748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 629/1252] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:09.220493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:09.378406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e95bf-796e-48b3-a61b-6d8e870257c2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:09.420628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:09.432040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:09.433426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:09.433948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:09.434439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:09.447777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:09.450260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:54:09.478163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:09.479236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 30 11:54:09.480195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:09.488215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 624/1253] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:09.489003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:09.556003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:09.610436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a0c52-0d89-4531-9cb0-d8f11c7449f2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:09.649946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:09.650670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:09.650975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:09.651291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:09.652211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:09.652531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:09.654450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:09.673176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 630/1254] 10.222.0.32 () {56 vars in 1143 bytes} [Mon May 30 11:54:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:54:09.679430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:09.791004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ca2e0-978a-4bdf-ab15-635aa95eb290 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:09.830522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8a4ab-dc0b-4c6c-9206-0359d8b222bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:09.831304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8a4ab-dc0b-4c6c-9206-0359d8b222bb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:09.831639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8a4ab-dc0b-4c6c-9206-0359d8b222bb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:09.831927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8a4ab-dc0b-4c6c-9206-0359d8b222bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:09.891477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:09.907454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8a4ab-dc0b-4c6c-9206-0359d8b222bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:09.909094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8a4ab-dc0b-4c6c-9206-0359d8b222bb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:10.020760 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 625/1255] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:54:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 30 11:54:10.027067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:10.180059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe432e2-3fbb-4ea2-833f-f3fea91e2aa5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:10.197451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:10.222995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:10.224227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:10.224629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:10.225110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:10.238263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:10.239852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:10.262931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:10.266106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:10.267168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:10.267168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:10.267168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:10.267168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:10.267168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 631/1256] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:10.271362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8a4ab-dc0b-4c6c-9206-0359d8b222bb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:10.397326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:10.410861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8a4ab-dc0b-4c6c-9206-0359d8b222bb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:10.450187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:10.450784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:10.451069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:10.451942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:10.452741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:10.453055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:10.454466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:10.467442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 626/1257] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:54:10.473939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:10.585740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42497f-8bdf-4d6b-94c2-fc448a832f99 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:10.622715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:10.624101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:10.624435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:10.624695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:10.632378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:10.633904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:54:10.650158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:10.651316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:10.652297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 632/1258] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:10.658075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:10.770180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddc864-5bab-491d-881b-34543af618eb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:10.805303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170db110-5ac3-4270-9495-6268d6237e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:10.805979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170db110-5ac3-4270-9495-6268d6237e30 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:10.806261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170db110-5ac3-4270-9495-6268d6237e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:10.806542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170db110-5ac3-4270-9495-6268d6237e30 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:10.807340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170db110-5ac3-4270-9495-6268d6237e30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:10.807653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170db110-5ac3-4270-9495-6268d6237e30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:10.809124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170db110-5ac3-4270-9495-6268d6237e30 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:10.821899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 627/1259] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:54:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:10.829150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:10.856823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:10.938594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769560e-5dea-41de-bd88-59a6d6c245e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:10.974494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314fa0e-7a38-4f64-ba80-168e2aa48691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:10.975159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314fa0e-7a38-4f64-ba80-168e2aa48691 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:10.975450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314fa0e-7a38-4f64-ba80-168e2aa48691 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:10.975734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314fa0e-7a38-4f64-ba80-168e2aa48691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:10.976609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314fa0e-7a38-4f64-ba80-168e2aa48691 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=6c654dad1f844fe9995c594116619216) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:10.976930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314fa0e-7a38-4f64-ba80-168e2aa48691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:10.978371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314fa0e-7a38-4f64-ba80-168e2aa48691 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:10.984404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 633/1260] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:54:10 2022] GET /identity/v3/role_assignments?user.id=6c654dad1f844fe9995c594116619216&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=4d21498572b14478a996ac538be5382e => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:13.215636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c02b-b10d-4c9e-8fc0-0c3b254477cd None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:13.216188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c02b-b10d-4c9e-8fc0-0c3b254477cd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:13.216188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c02b-b10d-4c9e-8fc0-0c3b254477cd None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:13.218463 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 628/1261] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:54:13.223702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45014-00f8-4e0a-a856-4155532e16d4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:13.224196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45014-00f8-4e0a-a856-4155532e16d4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:13.224290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a45014-00f8-4e0a-a856-4155532e16d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:13.242733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-98a45014-00f8-4e0a-a856-4155532e16d4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:54:13.243936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-98a45014-00f8-4e0a-a856-4155532e16d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:54:13.321827 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 634/1262] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:54:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 30 11:54:13.347251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90c02b-b10d-4c9e-8fc0-0c3b254477cd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:13.359244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:13.450653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90c02b-b10d-4c9e-8fc0-0c3b254477cd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:13.485037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0bcf2-80df-49d3-bc55-8ee76ee32ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:13.485752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0bcf2-80df-49d3-bc55-8ee76ee32ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:13.486045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0bcf2-80df-49d3-bc55-8ee76ee32ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:13.486325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0bcf2-80df-49d3-bc55-8ee76ee32ec2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:13.496289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0bcf2-80df-49d3-bc55-8ee76ee32ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-01a0bcf2-80df-49d3-bc55-8ee76ee32ec2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:54:13.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:13.505677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:13.506711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 629/1263] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:13.511869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a45014-00f8-4e0a-a856-4155532e16d4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:13.530799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:13.611480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a45014-00f8-4e0a-a856-4155532e16d4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:13.645513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:13.646176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:13.646461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:13.646736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:13.647486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:13.647802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:13.649206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:13.657364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 635/1264] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:13.663189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0bcf2-80df-49d3-bc55-8ee76ee32ec2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:13.729048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:13.763578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0bcf2-80df-49d3-bc55-8ee76ee32ec2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:13.797595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:13.798373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:13.798655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:13.798929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:13.809121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:13.811175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:13.828231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:13.829278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:13.830218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:13.830218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:13.830218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:13.830218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:13.830218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 630/1265] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:13.835122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:13.916269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:13.937221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c5fdb-6d6b-46e2-8fd9-780e32b2b237 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:13.975533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:13.976215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:13.976503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:13.976821 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:13.977498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:13.977808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:13.979175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:13.991252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 636/1266] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:13.996948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:14.097317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcfde6-f161-4e81-9791-52e36b072141 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:14.117068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:14.131646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:14.132475 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:14.132762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:14.133038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:14.140897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:14.142336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:54:14.157351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:14.158596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:14.159607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 631/1267] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:14.164520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:14.262344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fe8f0-cb9e-4f75-a76e-0b41a422f091 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:14.302646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:14.303389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:14.303720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:14.304003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:14.304890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:14.305205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:14.306642 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:14.315871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:14.324933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 637/1268] 10.222.0.32 () {56 vars in 1143 bytes} [Mon May 30 11:54:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:14.331004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:14.430924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348c1a3-f477-4d31-92c5-edb07f54657c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:14.463208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:14.463992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:14.464299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:14.464576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:14.476244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:14.477683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:14.495128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:14.496240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:14.497601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:14.497601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:14.497601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:14.497601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:14.497601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 632/1269] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:14.502248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:14.602528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb303-de71-472e-b605-0c92dd9e362e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:14.635809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:14.636500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:14.636789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:14.637070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:14.637792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:14.638107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:14.640120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:14.651578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 638/1270] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:14.657763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:14.689390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:14.758521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ce3c1-abe1-47ba-861b-9696fa04b68b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:14.791313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:14.792096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:14.792408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:14.792692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:14.799811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:14.801280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:54:14.815804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:14.816723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:14.817689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 633/1271] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:14.822939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:14.881610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:14.924007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdb0bd-2c44-4827-8a64-01697dcf7e8c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:14.956730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3c60-102c-42ca-918d-18dc07b27f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:14.957320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3c60-102c-42ca-918d-18dc07b27f5f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:14.957607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3c60-102c-42ca-918d-18dc07b27f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:14.957887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3c60-102c-42ca-918d-18dc07b27f5f None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:14.958671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3c60-102c-42ca-918d-18dc07b27f5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:14.958988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3c60-102c-42ca-918d-18dc07b27f5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:14.960456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3c60-102c-42ca-918d-18dc07b27f5f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:14.973927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 639/1272] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:14.980049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:15.071228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:15.080288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c254b3-efad-408e-812b-fbf46e8d4a6e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:15.114626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dbd32-22b9-4146-adfd-dbe3ee31e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:15.115521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dbd32-22b9-4146-adfd-dbe3ee31e3e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:15.115810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dbd32-22b9-4146-adfd-dbe3ee31e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:15.116092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dbd32-22b9-4146-adfd-dbe3ee31e3e4 None admin] PATH_INFO: `/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/6c654dad1f844fe9995c594116619216/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:15.118074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dbd32-22b9-4146-adfd-dbe3ee31e3e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccd54f1160f441ff95c6a7a1efc923df, user_id=6c654dad1f844fe9995c594116619216, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:15.119516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dbd32-22b9-4146-adfd-dbe3ee31e3e4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:15.132427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 634/1273] 10.222.0.32 () {58 vars in 1439 bytes} [Mon May 30 11:54:14 2022] PUT /identity/v3/projects/ccd54f1160f441ff95c6a7a1efc923df/users/6c654dad1f844fe9995c594116619216/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 11:54:17.408899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a373fd-51ae-4b0d-a61e-5826e9d2af1f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:17.409433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a373fd-51ae-4b0d-a61e-5826e9d2af1f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:17.409524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a373fd-51ae-4b0d-a61e-5826e9d2af1f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:17.410826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 640/1274] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:54:17.416040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17804-0268-4bbb-b112-0157a942bbb3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:17.416558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17804-0268-4bbb-b112-0157a942bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:17.416679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff17804-0268-4bbb-b112-0157a942bbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:17.433360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0ff17804-0268-4bbb-b112-0157a942bbb3 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:54:17.434553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ff17804-0268-4bbb-b112-0157a942bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:54:17.501016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 635/1275] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:54:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:54:17.524495 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a373fd-51ae-4b0d-a61e-5826e9d2af1f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:17.634260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a373fd-51ae-4b0d-a61e-5826e9d2af1f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:17.651227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:17.667829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f985e-c65a-42bc-a8e6-a0701945a4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:17.668598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f985e-c65a-42bc-a8e6-a0701945a4fb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:17.668889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f985e-c65a-42bc-a8e6-a0701945a4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:17.669168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f985e-c65a-42bc-a8e6-a0701945a4fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:17.679807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f985e-c65a-42bc-a8e6-a0701945a4fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-214f985e-c65a-42bc-a8e6-a0701945a4fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 30 11:54:17.688259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:17.689354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:17.690660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 641/1276] 10.222.0.32 () {56 vars in 1096 bytes} [Mon May 30 11:54:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:17.695513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff17804-0268-4bbb-b112-0157a942bbb3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:17.774142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:17.792958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff17804-0268-4bbb-b112-0157a942bbb3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:17.827395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:17.828061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:17.828372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:17.828652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:17.829402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:17.829711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:17.831075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:17.838901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 636/1277] 10.222.0.32 () {56 vars in 1099 bytes} [Mon May 30 11:54:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 11:54:17.845390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f985e-c65a-42bc-a8e6-a0701945a4fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:17.946818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f985e-c65a-42bc-a8e6-a0701945a4fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:17.980033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:17.980844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:17.981128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:17.981405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:17.992571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:17.994038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:18.010712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:18.011762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:18.012780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:18.012780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:18.012780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:18.012780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:18.012780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 642/1278] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:18.017943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:18.118134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e636d-a13d-4478-8a34-d52cb96e134c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:18.150595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:18.151257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:18.151542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:18.151819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:18.152569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:18.152882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:18.154771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:18.166450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 637/1279] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:18.172727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:18.202757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:18.272809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626d75-c567-4a82-b952-9a3778f52c11 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:18.312141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:18.313083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:18.313467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:18.313686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:18.322502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:18.324226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 30 11:54:18.341313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:18.342481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:18.343870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 643/1280] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:18.349159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:18.352964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:18.447256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819b18-e7d1-4f57-82fd-81c35cff3432 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:18.481545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:18.482261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:18.482751 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:18.482751 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:18.483569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:18.483883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:18.485345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:18.502216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 638/1281] 10.222.0.32 () {56 vars in 1143 bytes} [Mon May 30 11:54:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:18.507925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:18.598986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:18.609413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6eb78-2723-4018-8d9c-057b10e03c10 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:18.643111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e090b05-5fd2-4f0f-b7d5-e5ea6bf6bed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:18.643744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e090b05-5fd2-4f0f-b7d5-e5ea6bf6bed3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:18.644026 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e090b05-5fd2-4f0f-b7d5-e5ea6bf6bed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:18.644333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e090b05-5fd2-4f0f-b7d5-e5ea6bf6bed3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:18.710935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e090b05-5fd2-4f0f-b7d5-e5ea6bf6bed3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:18.712428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e090b05-5fd2-4f0f-b7d5-e5ea6bf6bed3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:18.801567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:18.812862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 644/1282] 10.222.0.32 () {58 vars in 1282 bytes} [Mon May 30 11:54:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 30 11:54:18.818616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:18.885564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:18.931078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457845e-89ab-455b-9a5e-7aa58c259460 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:18.964521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:18.965316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:18.965600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:18.965877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:18.976298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:18.977733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 30 11:54:18.995407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:18.996520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:18.997639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 30 11:54:18.997639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 30 11:54:18.997639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 30 11:54:18.997639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:18.997639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 639/1283] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:19.002348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e090b05-5fd2-4f0f-b7d5-e5ea6bf6bed3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:19.099711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e090b05-5fd2-4f0f-b7d5-e5ea6bf6bed3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:19.132633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:19.133292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:19.133579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:19.133856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:19.134579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:19.134884 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:19.136287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:19.148202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 645/1284] 10.222.0.32 () {56 vars in 1109 bytes} [Mon May 30 11:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:19.154608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:19.256907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6215207-2e80-4674-94e5-b73f08ea7115 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:19.272384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:19.296295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:19.297098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:19.297381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:19.297655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:19.304363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:19.305800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) May 30 11:54:19.319196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:19.320296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application May 30 11:54:19.321493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 640/1285] 10.222.0.32 () {56 vars in 1111 bytes} [Mon May 30 11:54:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:19.326873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:19.375840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:19.425471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a7488-54fc-484e-8d1f-3d3b83f698af None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:19.458819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f93f4c-9a70-4988-8f40-7d3714ed58a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:19.459469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93f4c-9a70-4988-8f40-7d3714ed58a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:19.459752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93f4c-9a70-4988-8f40-7d3714ed58a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:19.460028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93f4c-9a70-4988-8f40-7d3714ed58a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:19.460834 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f93f4c-9a70-4988-8f40-7d3714ed58a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:19.461145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f93f4c-9a70-4988-8f40-7d3714ed58a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:19.462574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f93f4c-9a70-4988-8f40-7d3714ed58a9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:19.474227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 646/1286] 10.222.0.32 () {56 vars in 1148 bytes} [Mon May 30 11:54:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:19.480254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:19.580350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09735311-3a64-4a2e-84b8-f01cf497340e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:19.613168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859f5a8-05a1-459d-9d10-c177e167e117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:19.613822 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859f5a8-05a1-459d-9d10-c177e167e117 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:19.614117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859f5a8-05a1-459d-9d10-c177e167e117 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:19.614398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859f5a8-05a1-459d-9d10-c177e167e117 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:19.615225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859f5a8-05a1-459d-9d10-c177e167e117 None admin] RBAC: Adding query filter params (role.id=4d21498572b14478a996ac538be5382e, scope.project.id=ccd54f1160f441ff95c6a7a1efc923df, user.id=6c654dad1f844fe9995c594116619216) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:19.615543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859f5a8-05a1-459d-9d10-c177e167e117 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:19.617005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859f5a8-05a1-459d-9d10-c177e167e117 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:19.623254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 641/1287] 10.222.0.32 () {56 vars in 1374 bytes} [Mon May 30 11:54:19 2022] GET /identity/v3/role_assignments?user.id=6c654dad1f844fe9995c594116619216&scope.project.id=ccd54f1160f441ff95c6a7a1efc923df&role.id=4d21498572b14478a996ac538be5382e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:21.881926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e608-3836-4681-85c7-ba8d12dcaff4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:21.882477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e608-3836-4681-85c7-ba8d12dcaff4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:21.882477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253e608-3836-4681-85c7-ba8d12dcaff4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:21.883780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 647/1288] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:54:21.889908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23af76-ae9c-452d-a85a-dfe6de38d369 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:21.890635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23af76-ae9c-452d-a85a-dfe6de38d369 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:21.890746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23af76-ae9c-452d-a85a-dfe6de38d369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:21.907330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-be23af76-ae9c-452d-a85a-dfe6de38d369 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:54:21.908538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-be23af76-ae9c-452d-a85a-dfe6de38d369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:54:21.938420 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:21.976645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 642/1289] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:54:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:54:21.999960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253e608-3836-4681-85c7-ba8d12dcaff4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:22.100783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253e608-3836-4681-85c7-ba8d12dcaff4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:22.111933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:22.135668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:22.136432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:22.136720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:22.137002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:22.137658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:22.139362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:22.151356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:22.152803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:22.154076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 648/1290] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:54:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:22.158824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23af76-ae9c-452d-a85a-dfe6de38d369 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:22.256757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23af76-ae9c-452d-a85a-dfe6de38d369 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:22.296565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94604b2-25cc-4619-b095-c68e391fe453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:22.297355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94604b2-25cc-4619-b095-c68e391fe453 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:22.297764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94604b2-25cc-4619-b095-c68e391fe453 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:22.298000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94604b2-25cc-4619-b095-c68e391fe453 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:22.298856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94604b2-25cc-4619-b095-c68e391fe453 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:22.299236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94604b2-25cc-4619-b095-c68e391fe453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:22.300916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94604b2-25cc-4619-b095-c68e391fe453 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:22.307052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 643/1291] 10.222.0.32 () {56 vars in 1116 bytes} [Mon May 30 11:54:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:22.314286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:22.420915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a416cd-b60b-486b-94d8-5ea88bf2b68b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:22.454879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06545fc4-8f8b-42e0-81a5-443b2fa51992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:22.455543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06545fc4-8f8b-42e0-81a5-443b2fa51992 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:22.455833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06545fc4-8f8b-42e0-81a5-443b2fa51992 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:22.456112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06545fc4-8f8b-42e0-81a5-443b2fa51992 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:22.456850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06545fc4-8f8b-42e0-81a5-443b2fa51992 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:22.457163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06545fc4-8f8b-42e0-81a5-443b2fa51992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:22.458547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06545fc4-8f8b-42e0-81a5-443b2fa51992 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:22.462973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 649/1292] 10.222.0.32 () {56 vars in 1116 bytes} [Mon May 30 11:54:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:24.684352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8db421-4d4a-457b-b3b2-cec9e3a7cc05 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:24.685601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8db421-4d4a-457b-b3b2-cec9e3a7cc05 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:24.685601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8db421-4d4a-457b-b3b2-cec9e3a7cc05 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:24.686777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 644/1293] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:54:24.692083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2656269-671f-488e-9abb-9cb87cdc577c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:24.692552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2656269-671f-488e-9abb-9cb87cdc577c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:24.692678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2656269-671f-488e-9abb-9cb87cdc577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:24.710079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f2656269-671f-488e-9abb-9cb87cdc577c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:54:24.711269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f2656269-671f-488e-9abb-9cb87cdc577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:54:24.740019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:24.779028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 650/1294] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:54:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:54:24.802220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8db421-4d4a-457b-b3b2-cec9e3a7cc05 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:24.834771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:24.905947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8db421-4d4a-457b-b3b2-cec9e3a7cc05 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:24.938977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0030a9-0a2a-4a94-8530-febb353d7f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:24.939657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0030a9-0a2a-4a94-8530-febb353d7f14 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:24.939931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0030a9-0a2a-4a94-8530-febb353d7f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:24.940244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0030a9-0a2a-4a94-8530-febb353d7f14 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:24.940971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0030a9-0a2a-4a94-8530-febb353d7f14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:24.942337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0030a9-0a2a-4a94-8530-febb353d7f14 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:24.955976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 645/1295] 10.222.0.32 () {60 vars in 1141 bytes} [Mon May 30 11:54:24 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:54:27.235943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cdd46-516e-42d0-8717-12a27410d917 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:27.236542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cdd46-516e-42d0-8717-12a27410d917 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:27.236542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221cdd46-516e-42d0-8717-12a27410d917 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:27.237918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 651/1296] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:54:27.243098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9411c2-12d1-42be-822a-0be858791fe2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:27.243512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9411c2-12d1-42be-822a-0be858791fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:27.243652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9411c2-12d1-42be-822a-0be858791fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:27.260850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1c9411c2-12d1-42be-822a-0be858791fe2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:54:27.262032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1c9411c2-12d1-42be-822a-0be858791fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:54:27.354688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 646/1297] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:54:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:54:27.378837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cdd46-516e-42d0-8717-12a27410d917 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:27.610169 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221cdd46-516e-42d0-8717-12a27410d917 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:27.643092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:27.643877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:27.644170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:27.644454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:27.645100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:27.646450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:27.658771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:27.659937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:27.661199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 652/1298] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:54:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:27.666078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9411c2-12d1-42be-822a-0be858791fe2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:27.696816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:27.783203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9411c2-12d1-42be-822a-0be858791fe2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:27.816428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:27.817096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:27.817360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:27.817631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:27.818325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:27.818630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:27.819982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:27.824969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 647/1299] 10.222.0.32 () {56 vars in 1116 bytes} [Mon May 30 11:54:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:27.831122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:27.877200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:27.944501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd05d-94de-4857-97b4-f0a16ca007ff None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:27.978220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe918ef9-4500-4408-9e42-c7c1cbf75892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:27.978876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe918ef9-4500-4408-9e42-c7c1cbf75892 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:27.979159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe918ef9-4500-4408-9e42-c7c1cbf75892 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:27.979434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe918ef9-4500-4408-9e42-c7c1cbf75892 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:27.980237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe918ef9-4500-4408-9e42-c7c1cbf75892 None admin] RBAC: Adding query filter params (interface=public, service_id=542e58880784464aa30e18fca2d66da2, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:27.980551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe918ef9-4500-4408-9e42-c7c1cbf75892 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:27.982577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe918ef9-4500-4408-9e42-c7c1cbf75892 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:27.987558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 653/1300] 10.222.0.32 () {56 vars in 1251 bytes} [Mon May 30 11:54:27 2022] GET /identity/v3/endpoints?service_id=542e58880784464aa30e18fca2d66da2&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:27.993249 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:28.087857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:28.108554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c221d9-b721-4ed5-b215-aa1e20cc38c2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:28.143418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2c7f3-2231-4315-9ca6-e5ec7ebc670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:28.144012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2c7f3-2231-4315-9ca6-e5ec7ebc670d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:28.144332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2c7f3-2231-4315-9ca6-e5ec7ebc670d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:28.144616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2c7f3-2231-4315-9ca6-e5ec7ebc670d None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:28.145299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2c7f3-2231-4315-9ca6-e5ec7ebc670d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:28.146666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2c7f3-2231-4315-9ca6-e5ec7ebc670d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:28.153350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 648/1301] 10.222.0.32 () {56 vars in 1087 bytes} [Mon May 30 11:54:27 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 11:54:30.404893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436bd9-6327-4848-af9a-b0e9f491390c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:30.405455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436bd9-6327-4848-af9a-b0e9f491390c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:30.405455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef436bd9-6327-4848-af9a-b0e9f491390c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:30.406812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 654/1302] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:54:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:54:30.412011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e1e26-a0cc-4019-8d12-a3f5ed1fb76f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:30.412469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e1e26-a0cc-4019-8d12-a3f5ed1fb76f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:30.412602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e1e26-a0cc-4019-8d12-a3f5ed1fb76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:30.428919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-196e1e26-a0cc-4019-8d12-a3f5ed1fb76f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:54:30.430112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-196e1e26-a0cc-4019-8d12-a3f5ed1fb76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:54:30.499779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 649/1303] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:54:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 30 11:54:30.522843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef436bd9-6327-4848-af9a-b0e9f491390c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:30.617723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:30.625559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef436bd9-6327-4848-af9a-b0e9f491390c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:30.659037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:30.659766 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:30.660052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:30.660369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] PATH_INFO: `/v3/services/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:30.661020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:30.662565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 30 11:54:30.675832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 30 11:54:30.677020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 30 11:54:30.677906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 655/1304] 10.222.0.32 () {56 vars in 1117 bytes} [Mon May 30 11:54:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 30 11:54:30.683254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e1e26-a0cc-4019-8d12-a3f5ed1fb76f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:30.766236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:30.790310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e1e26-a0cc-4019-8d12-a3f5ed1fb76f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:30.827564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7085d6-b850-454e-99e7-5a348b4f9eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:30.828254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7085d6-b850-454e-99e7-5a348b4f9eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:30.828536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7085d6-b850-454e-99e7-5a348b4f9eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:30.828809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7085d6-b850-454e-99e7-5a348b4f9eb5 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:30.829511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7085d6-b850-454e-99e7-5a348b4f9eb5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 11:54:30.829818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7085d6-b850-454e-99e7-5a348b4f9eb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:30.831177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7085d6-b850-454e-99e7-5a348b4f9eb5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:30.836236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 650/1305] 10.222.0.32 () {56 vars in 1116 bytes} [Mon May 30 11:54:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 11:54:30.842214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:54:30.942329 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708f1e9-909b-4dfe-8f21-6b470ffd1fff None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:54:30.975611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1926c7c-40d0-494e-adfb-a1abf34eccaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:54:30.976294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926c7c-40d0-494e-adfb-a1abf34eccaf None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:54:30.976584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926c7c-40d0-494e-adfb-a1abf34eccaf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:54:30.976867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1926c7c-40d0-494e-adfb-a1abf34eccaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:54:30.977602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1926c7c-40d0-494e-adfb-a1abf34eccaf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:54:30.978974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1926c7c-40d0-494e-adfb-a1abf34eccaf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:54:30.997911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 656/1306] 10.222.0.32 () {60 vars in 1144 bytes} [Mon May 30 11:54:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 11:54:40.833635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=30 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:54:40.993257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:36.361906 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e995-7b62-4d45-82a7-fde2310402a1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:36.362497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e995-7b62-4d45-82a7-fde2310402a1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:36.362497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e995-7b62-4d45-82a7-fde2310402a1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:36.363870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 651/1307] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:36.369798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dddf4-54ee-4769-a10a-89d2bd02b3df None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:36.370443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dddf4-54ee-4769-a10a-89d2bd02b3df None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:36.370443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dddf4-54ee-4769-a10a-89d2bd02b3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:36.449850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f39dddf4-54ee-4769-a10a-89d2bd02b3df None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:36.451046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f39dddf4-54ee-4769-a10a-89d2bd02b3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:36.563026 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 657/1308] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:36.613055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09d3c0-532a-47d3-b13c-ace504d38a53 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:36.613396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09d3c0-532a-47d3-b13c-ace504d38a53 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:36.613702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09d3c0-532a-47d3-b13c-ace504d38a53 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:36.615261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 652/1309] 10.222.0.32 () {54 vars in 964 bytes} [Mon May 30 11:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:36.621936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2049a-bda2-45e4-85cd-fbe810a2ff41 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:36.622226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2049a-bda2-45e4-85cd-fbe810a2ff41 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:36.622506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2049a-bda2-45e4-85cd-fbe810a2ff41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:36.717687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c4e2049a-bda2-45e4-85cd-fbe810a2ff41 None None] MFA Rules not processed for user `15ea38389eee4472a8917023643be9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:36.720856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c4e2049a-bda2-45e4-85cd-fbe810a2ff41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:36.747890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:36.856934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 658/1310] 10.222.0.32 () {58 vars in 1061 bytes} [Mon May 30 11:55:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:36.867545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09d3c0-532a-47d3-b13c-ace504d38a53 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:55:36.869891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0d09d3c0-532a-47d3-b13c-ace504d38a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:37.020072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09d3c0-532a-47d3-b13c-ace504d38a53 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:55:37.117548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d90d8-dc07-4e73-8964-0dfcc11a2c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:55:37.118210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d90d8-dc07-4e73-8964-0dfcc11a2c81 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:37.118496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d90d8-dc07-4e73-8964-0dfcc11a2c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:37.118773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d90d8-dc07-4e73-8964-0dfcc11a2c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:37.120932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-368d90d8-dc07-4e73-8964-0dfcc11a2c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:37.180925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:37.240095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d90d8-dc07-4e73-8964-0dfcc11a2c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:55:37.241577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d90d8-dc07-4e73-8964-0dfcc11a2c81 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:55:37.267289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 653/1311] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:55:39.848929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d843-6d39-4bce-ae67-bb3fe0d3a78d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:39.849412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d843-6d39-4bce-ae67-bb3fe0d3a78d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:39.849412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d843-6d39-4bce-ae67-bb3fe0d3a78d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:39.850799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 659/1312] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:39.857021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ad24a-e1b6-45e4-b6c1-04a447a1cee9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:39.857477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ad24a-e1b6-45e4-b6c1-04a447a1cee9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:39.857578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ad24a-e1b6-45e4-b6c1-04a447a1cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:39.947366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bc0ad24a-e1b6-45e4-b6c1-04a447a1cee9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:39.948605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bc0ad24a-e1b6-45e4-b6c1-04a447a1cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:40.051366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 654/1313] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:40.091609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6d843-6d39-4bce-ae67-bb3fe0d3a78d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:55:40.093208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55e6d843-6d39-4bce-ae67-bb3fe0d3a78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:40.202058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6d843-6d39-4bce-ae67-bb3fe0d3a78d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:55:40.279178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:40.281837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c36ea-9bc3-4f9c-8697-ee28ad5802d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:55:40.282810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c36ea-9bc3-4f9c-8697-ee28ad5802d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:40.283289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c36ea-9bc3-4f9c-8697-ee28ad5802d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:40.283797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c36ea-9bc3-4f9c-8697-ee28ad5802d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:40.285899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-568c36ea-9bc3-4f9c-8697-ee28ad5802d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:40.401639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c36ea-9bc3-4f9c-8697-ee28ad5802d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:55:40.403107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c36ea-9bc3-4f9c-8697-ee28ad5802d8 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:55:40.422745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 660/1314] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:55:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:55:43.061074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffdcd0-92a2-4276-b769-54c292402cf9 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:43.061602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffdcd0-92a2-4276-b769-54c292402cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:43.061602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffdcd0-92a2-4276-b769-54c292402cf9 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:43.062978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 655/1315] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:43.068257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3ffc6-8357-43b5-a439-2d5ba03e0f2e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:43.068678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3ffc6-8357-43b5-a439-2d5ba03e0f2e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:43.068836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3ffc6-8357-43b5-a439-2d5ba03e0f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:43.137890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e2a3ffc6-8357-43b5-a439-2d5ba03e0f2e None None] MFA Rules not processed for user `156da27655884b45a9652caa634889b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:43.139064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e2a3ffc6-8357-43b5-a439-2d5ba03e0f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:43.192769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:43.243162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 661/1316] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:43.283973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffdcd0-92a2-4276-b769-54c292402cf9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:55:43.285681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-89ffdcd0-92a2-4276-b769-54c292402cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:43.297339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:43.396660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffdcd0-92a2-4276-b769-54c292402cf9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:55:43.473645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8277f8-a8bd-4c53-b495-b5e226359169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:55:43.474304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8277f8-a8bd-4c53-b495-b5e226359169 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:43.474591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8277f8-a8bd-4c53-b495-b5e226359169 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:43.474868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8277f8-a8bd-4c53-b495-b5e226359169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:43.476565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ab8277f8-a8bd-4c53-b495-b5e226359169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:43.544100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:43.594138 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8277f8-a8bd-4c53-b495-b5e226359169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:55:43.595583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8277f8-a8bd-4c53-b495-b5e226359169 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:55:43.616701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 656/1317] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:55:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:55:46.484180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b295937-7603-4098-8bdb-b7eb222f7fc1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:46.484664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b295937-7603-4098-8bdb-b7eb222f7fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:46.484764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b295937-7603-4098-8bdb-b7eb222f7fc1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:46.486132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 662/1318] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:46.491462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27bb64-11eb-49f8-8f6c-84b75d835532 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:46.491876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27bb64-11eb-49f8-8f6c-84b75d835532 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:46.492031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27bb64-11eb-49f8-8f6c-84b75d835532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:46.688351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7f27bb64-11eb-49f8-8f6c-84b75d835532 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:46.689559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7f27bb64-11eb-49f8-8f6c-84b75d835532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:46.788636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 657/1319] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:55:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:46.827663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b295937-7603-4098-8bdb-b7eb222f7fc1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:55:46.829241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6b295937-7603-4098-8bdb-b7eb222f7fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:46.933959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b295937-7603-4098-8bdb-b7eb222f7fc1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:55:47.006628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2f7da-cafd-46be-9b2c-d79a28a2f19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:55:47.007285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2f7da-cafd-46be-9b2c-d79a28a2f19e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:47.007570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2f7da-cafd-46be-9b2c-d79a28a2f19e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:47.007850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2f7da-cafd-46be-9b2c-d79a28a2f19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:47.009548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f6b2f7da-cafd-46be-9b2c-d79a28a2f19e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:47.017245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:47.120748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2f7da-cafd-46be-9b2c-d79a28a2f19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:55:47.122107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2f7da-cafd-46be-9b2c-d79a28a2f19e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:55:47.140141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 663/1320] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:55:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:55:50.109373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aefe3-5a64-4daa-b8c5-9d2b41facb7f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:50.109938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aefe3-5a64-4daa-b8c5-9d2b41facb7f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:50.109938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aefe3-5a64-4daa-b8c5-9d2b41facb7f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:50.111305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 658/1321] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:50.116576 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e260ecb-c0ad-4e90-a35c-e33787202872 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:50.116993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e260ecb-c0ad-4e90-a35c-e33787202872 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:50.117145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e260ecb-c0ad-4e90-a35c-e33787202872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:50.186361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6e260ecb-c0ad-4e90-a35c-e33787202872 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:50.187570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6e260ecb-c0ad-4e90-a35c-e33787202872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:50.245952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:50.287251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 664/1322] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:50.323725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1aefe3-5a64-4daa-b8c5-9d2b41facb7f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:55:50.325356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7c1aefe3-5a64-4daa-b8c5-9d2b41facb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:50.345612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:50.433933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1aefe3-5a64-4daa-b8c5-9d2b41facb7f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:55:50.517707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0706ed-e2d5-4401-9870-8422ba678cb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:55:50.518369 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0706ed-e2d5-4401-9870-8422ba678cb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:50.518653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0706ed-e2d5-4401-9870-8422ba678cb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:50.518931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0706ed-e2d5-4401-9870-8422ba678cb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:50.520610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7f0706ed-e2d5-4401-9870-8422ba678cb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:50.595531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:50.638296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0706ed-e2d5-4401-9870-8422ba678cb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:55:50.639984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0706ed-e2d5-4401-9870-8422ba678cb5 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:55:50.658958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 659/1323] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:55:53.028542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1041b-f478-47ee-be59-5f9c482213f1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:53.029189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1041b-f478-47ee-be59-5f9c482213f1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:53.029189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1041b-f478-47ee-be59-5f9c482213f1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:53.030460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 665/1324] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:53.035995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55564a-bcb4-4b90-9a1d-4470731a5a27 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:53.036506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55564a-bcb4-4b90-9a1d-4470731a5a27 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:53.036610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55564a-bcb4-4b90-9a1d-4470731a5a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:53.102691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9b55564a-bcb4-4b90-9a1d-4470731a5a27 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:53.103884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9b55564a-bcb4-4b90-9a1d-4470731a5a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:53.204336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 660/1325] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:53.242040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1041b-f478-47ee-be59-5f9c482213f1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:55:53.243647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-49e1041b-f478-47ee-be59-5f9c482213f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:53.350916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1041b-f478-47ee-be59-5f9c482213f1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:55:53.425189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521c6d1-d097-4410-93fd-f3486ea8ec8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:55:53.425851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521c6d1-d097-4410-93fd-f3486ea8ec8e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:53.426135 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521c6d1-d097-4410-93fd-f3486ea8ec8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:53.426418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521c6d1-d097-4410-93fd-f3486ea8ec8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:53.428064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2521c6d1-d097-4410-93fd-f3486ea8ec8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:53.444955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:53.544338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521c6d1-d097-4410-93fd-f3486ea8ec8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:55:53.545824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521c6d1-d097-4410-93fd-f3486ea8ec8e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:55:53.564477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 666/1326] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:55:55.916390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8112-3fc1-45ca-9933-b3c32796a5dc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:55.917511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8112-3fc1-45ca-9933-b3c32796a5dc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:55.917511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8112-3fc1-45ca-9933-b3c32796a5dc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:55.919202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 661/1327] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:55.924768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b63ed3-5b31-437f-a9ce-68e84970fdc7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:55.925209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b63ed3-5b31-437f-a9ce-68e84970fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:55.925313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b63ed3-5b31-437f-a9ce-68e84970fdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:55.996987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e2b63ed3-5b31-437f-a9ce-68e84970fdc7 None None] MFA Rules not processed for user `156da27655884b45a9652caa634889b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:55.998188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e2b63ed3-5b31-437f-a9ce-68e84970fdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:56.067989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:56.100029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 667/1328] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:55:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:56.138526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b8112-3fc1-45ca-9933-b3c32796a5dc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:55:56.140425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-057b8112-3fc1-45ca-9933-b3c32796a5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:56.174512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:56.250302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b8112-3fc1-45ca-9933-b3c32796a5dc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:55:56.325439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498a8cf-aa70-49e0-9e59-2fc830a86496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:55:56.326105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498a8cf-aa70-49e0-9e59-2fc830a86496 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:56.326421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498a8cf-aa70-49e0-9e59-2fc830a86496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:56.326727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498a8cf-aa70-49e0-9e59-2fc830a86496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:56.328438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4498a8cf-aa70-49e0-9e59-2fc830a86496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:56.407457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:56.441025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498a8cf-aa70-49e0-9e59-2fc830a86496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:55:56.442386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498a8cf-aa70-49e0-9e59-2fc830a86496 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:55:56.461213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 662/1329] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:55:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:55:58.978102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed53b-3a66-4351-ac94-267962595576 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:58.980754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed53b-3a66-4351-ac94-267962595576 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:58.981046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed53b-3a66-4351-ac94-267962595576 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:58.982563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 668/1330] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:55:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:55:58.988315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c581c-13e3-4b7b-b392-47df39795418 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:58.988779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c581c-13e3-4b7b-b392-47df39795418 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:58.988894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c581c-13e3-4b7b-b392-47df39795418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:59.068112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-da1c581c-13e3-4b7b-b392-47df39795418 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:55:59.069370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-da1c581c-13e3-4b7b-b392-47df39795418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:59.167220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 663/1331] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:55:59.204889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ed53b-3a66-4351-ac94-267962595576 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:55:59.206421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-064ed53b-3a66-4351-ac94-267962595576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:59.311650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ed53b-3a66-4351-ac94-267962595576 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:55:59.388405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4df13e-e59c-4a66-a5cc-dbc4bf33a662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:55:59.389064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df13e-e59c-4a66-a5cc-dbc4bf33a662 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:55:59.389351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df13e-e59c-4a66-a5cc-dbc4bf33a662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:55:59.389626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df13e-e59c-4a66-a5cc-dbc4bf33a662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:55:59.391539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c4df13e-e59c-4a66-a5cc-dbc4bf33a662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:55:59.419128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:55:59.501898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4df13e-e59c-4a66-a5cc-dbc4bf33a662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:55:59.503356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4df13e-e59c-4a66-a5cc-dbc4bf33a662 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:55:59.521403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 669/1332] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:55:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:01.877618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804b343-cf54-4230-b5e8-ca941cd7c398 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:01.878151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804b343-cf54-4230-b5e8-ca941cd7c398 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:01.878151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804b343-cf54-4230-b5e8-ca941cd7c398 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:01.879482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 664/1333] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:01.884973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807bc93-0caa-493b-90dd-3f93ec6e15d9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:01.885573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807bc93-0caa-493b-90dd-3f93ec6e15d9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:01.885573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807bc93-0caa-493b-90dd-3f93ec6e15d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:01.950403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d807bc93-0caa-493b-90dd-3f93ec6e15d9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:01.951645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d807bc93-0caa-493b-90dd-3f93ec6e15d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:02.031948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:02.055480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 670/1334] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:02.096561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804b343-cf54-4230-b5e8-ca941cd7c398 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:02.098087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6804b343-cf54-4230-b5e8-ca941cd7c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:02.138224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:02.203233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804b343-cf54-4230-b5e8-ca941cd7c398 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:02.276801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0f50a-eee5-4da1-a9c1-aa09e4f3726b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:02.277447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0f50a-eee5-4da1-a9c1-aa09e4f3726b service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:02.277736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0f50a-eee5-4da1-a9c1-aa09e4f3726b service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:02.278036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0f50a-eee5-4da1-a9c1-aa09e4f3726b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:02.279693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5eb0f50a-eee5-4da1-a9c1-aa09e4f3726b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:02.362701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:02.388786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0f50a-eee5-4da1-a9c1-aa09e4f3726b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:02.390420 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0f50a-eee5-4da1-a9c1-aa09e4f3726b service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:02.409268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 665/1335] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:05.176803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6143c1-27a6-4517-bbf9-fc644d933872 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:05.177381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6143c1-27a6-4517-bbf9-fc644d933872 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:05.177381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6143c1-27a6-4517-bbf9-fc644d933872 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:05.178731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 671/1336] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:05.184648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc493-aa35-45ce-8552-24521e5e5fff None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:05.185085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc493-aa35-45ce-8552-24521e5e5fff None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:05.185212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc493-aa35-45ce-8552-24521e5e5fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:05.249159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8acbc493-aa35-45ce-8552-24521e5e5fff None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:05.250357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8acbc493-aa35-45ce-8552-24521e5e5fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:05.349247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 666/1337] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:05.386046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6143c1-27a6-4517-bbf9-fc644d933872 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:05.387836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ea6143c1-27a6-4517-bbf9-fc644d933872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:05.497798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6143c1-27a6-4517-bbf9-fc644d933872 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:05.572753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd1859-87b2-485e-90fd-17205d3063bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:05.573807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd1859-87b2-485e-90fd-17205d3063bd service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:05.574056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd1859-87b2-485e-90fd-17205d3063bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:05.574342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd1859-87b2-485e-90fd-17205d3063bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:05.576074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4bcd1859-87b2-485e-90fd-17205d3063bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:05.612631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:05.689100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd1859-87b2-485e-90fd-17205d3063bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:05.690543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd1859-87b2-485e-90fd-17205d3063bd service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:05.708615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 672/1338] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 11:56:10.020818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07d246-937d-4e6b-83c9-166488c5b948 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:10.021325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07d246-937d-4e6b-83c9-166488c5b948 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:10.021325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07d246-937d-4e6b-83c9-166488c5b948 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:10.022650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 667/1339] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:10.028598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe38e-e3d6-4c54-a24f-8d0389ea7103 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:10.029087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe38e-e3d6-4c54-a24f-8d0389ea7103 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:10.029628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe38e-e3d6-4c54-a24f-8d0389ea7103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:10.105142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-876fe38e-e3d6-4c54-a24f-8d0389ea7103 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:10.106380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-876fe38e-e3d6-4c54-a24f-8d0389ea7103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:10.205360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:10.219698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 673/1340] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 11:56:10.274341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6c3e-c94f-4284-97e9-40d61f046723 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:10.274675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6c3e-c94f-4284-97e9-40d61f046723 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:10.275342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f6c3e-c94f-4284-97e9-40d61f046723 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:10.277449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 668/1341] 10.222.0.32 () {54 vars in 964 bytes} [Mon May 30 11:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:10.284339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2df65-2f57-46e2-8948-a4757b1d5c52 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:10.284708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2df65-2f57-46e2-8948-a4757b1d5c52 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:10.285091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2df65-2f57-46e2-8948-a4757b1d5c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:10.356599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f9b2df65-2f57-46e2-8948-a4757b1d5c52 None None] MFA Rules not processed for user `15ea38389eee4472a8917023643be9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:10.357816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9b2df65-2f57-46e2-8948-a4757b1d5c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:10.435153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 674/1342] 10.222.0.32 () {58 vars in 1061 bytes} [Mon May 30 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:10.444335 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6c3e-c94f-4284-97e9-40d61f046723 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:10.456860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f6c3e-c94f-4284-97e9-40d61f046723 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:10.460417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667b4a0-bf14-41d4-b374-c1f6c6a1ea0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:10.461059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667b4a0-bf14-41d4-b374-c1f6c6a1ea0c service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:10.461340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667b4a0-bf14-41d4-b374-c1f6c6a1ea0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:10.461615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667b4a0-bf14-41d4-b374-c1f6c6a1ea0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:10.471750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667b4a0-bf14-41d4-b374-c1f6c6a1ea0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:10.473310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667b4a0-bf14-41d4-b374-c1f6c6a1ea0c service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:10.485273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 669/1343] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:14.003728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857eb6-bb62-4793-9fc3-b8cbb326fa34 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:14.004448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857eb6-bb62-4793-9fc3-b8cbb326fa34 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:14.004448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857eb6-bb62-4793-9fc3-b8cbb326fa34 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:14.005635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 675/1344] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:14.011709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1146c2-0475-4191-9842-d589a6003374 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:14.012217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1146c2-0475-4191-9842-d589a6003374 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:14.012346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1146c2-0475-4191-9842-d589a6003374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:14.040988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8d1146c2-0475-4191-9842-d589a6003374 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:14.042483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8d1146c2-0475-4191-9842-d589a6003374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:14.064887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:14.128785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 670/1345] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:14.167029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857eb6-bb62-4793-9fc3-b8cbb326fa34 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:14.177722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857eb6-bb62-4793-9fc3-b8cbb326fa34 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:14.181274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed29187-384a-4ae2-a557-dc6afaeea1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:14.181926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29187-384a-4ae2-a557-dc6afaeea1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:14.182765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29187-384a-4ae2-a557-dc6afaeea1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:14.183050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29187-384a-4ae2-a557-dc6afaeea1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:14.192288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed29187-384a-4ae2-a557-dc6afaeea1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:14.193725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed29187-384a-4ae2-a557-dc6afaeea1b6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:14.203777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 676/1346] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:18.764427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe34a5-22ee-49c7-92d9-264a94d18b8d None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:18.764934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe34a5-22ee-49c7-92d9-264a94d18b8d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:18.764934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe34a5-22ee-49c7-92d9-264a94d18b8d None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:18.766709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 671/1347] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:18.772020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63db13-c1ec-436b-bbad-147339bd82e7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:18.772471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63db13-c1ec-436b-bbad-147339bd82e7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:18.772638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63db13-c1ec-436b-bbad-147339bd82e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:18.790516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ac63db13-c1ec-436b-bbad-147339bd82e7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:18.792379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ac63db13-c1ec-436b-bbad-147339bd82e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:18.800424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 677/1348] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:18.839228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe34a5-22ee-49c7-92d9-264a94d18b8d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:18.849765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe34a5-22ee-49c7-92d9-264a94d18b8d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:18.853391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094607b9-4ede-4303-89e8-9c4e31621714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:18.854519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094607b9-4ede-4303-89e8-9c4e31621714 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:18.854996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094607b9-4ede-4303-89e8-9c4e31621714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:18.855416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094607b9-4ede-4303-89e8-9c4e31621714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:18.864728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094607b9-4ede-4303-89e8-9c4e31621714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:18.866154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094607b9-4ede-4303-89e8-9c4e31621714 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:18.877004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 672/1349] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:24.325035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a9aa1-2cb1-4171-8374-d59f8b154abd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:24.325593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a9aa1-2cb1-4171-8374-d59f8b154abd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:24.325593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a9aa1-2cb1-4171-8374-d59f8b154abd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:24.327234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 678/1350] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 11:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:24.332229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ee8c6-ab69-4608-ba41-c6d3163cc411 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:24.333264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ee8c6-ab69-4608-ba41-c6d3163cc411 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:24.333765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ee8c6-ab69-4608-ba41-c6d3163cc411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:24.390453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7e5ee8c6-ab69-4608-ba41-c6d3163cc411 None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:24.392596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7e5ee8c6-ab69-4608-ba41-c6d3163cc411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:24.482969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 673/1351] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 11:56:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:28.785668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=66 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:28.929164 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcfdd2-fb50-4a2f-9cce-8623ebe7898f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:28.929579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcfdd2-fb50-4a2f-9cce-8623ebe7898f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:28.929810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcfdd2-fb50-4a2f-9cce-8623ebe7898f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:28.931292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 679/1352] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 11:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:28.936869 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8051e0-6319-403b-a67a-17de6b258727 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:28.937375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8051e0-6319-403b-a67a-17de6b258727 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:28.937459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8051e0-6319-403b-a67a-17de6b258727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:28.958692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dc8051e0-6319-403b-a67a-17de6b258727 None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:28.960122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dc8051e0-6319-403b-a67a-17de6b258727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:28.970712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 674/1353] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:30.583704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4059d19-ccc1-43e2-8202-15846021b28f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:30.584245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4059d19-ccc1-43e2-8202-15846021b28f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:30.584376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4059d19-ccc1-43e2-8202-15846021b28f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:30.585889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 680/1354] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 11:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:30.591544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc3c91-047b-405c-aaf7-4467edd14698 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:30.592103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc3c91-047b-405c-aaf7-4467edd14698 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:30.592437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc3c91-047b-405c-aaf7-4467edd14698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:30.614011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3cdc3c91-047b-405c-aaf7-4467edd14698 None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:30.616214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3cdc3c91-047b-405c-aaf7-4467edd14698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:30.629303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 675/1355] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:34.530088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051859bc-13f1-475b-88bb-744d9e4bb7bc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:34.530611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051859bc-13f1-475b-88bb-744d9e4bb7bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:34.530710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051859bc-13f1-475b-88bb-744d9e4bb7bc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:34.532350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 681/1356] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:34.538258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3908a7-73a0-4fc7-95fd-68479add7b62 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:34.539159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3908a7-73a0-4fc7-95fd-68479add7b62 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:34.539673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3908a7-73a0-4fc7-95fd-68479add7b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:34.560600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fd3908a7-73a0-4fc7-95fd-68479add7b62 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:34.561802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fd3908a7-73a0-4fc7-95fd-68479add7b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:34.569765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 676/1357] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:34.629491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdd32e-f1de-4bf3-b11b-6ad8b8166881 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:34.630023 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdd32e-f1de-4bf3-b11b-6ad8b8166881 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:34.630570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdd32e-f1de-4bf3-b11b-6ad8b8166881 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:34.633421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 682/1358] 10.222.0.32 () {54 vars in 961 bytes} [Mon May 30 11:56:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:34.645144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab87ac7-b833-4202-8f81-b5e357041a23 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:34.645908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab87ac7-b833-4202-8f81-b5e357041a23 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:34.646306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab87ac7-b833-4202-8f81-b5e357041a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:34.706480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0ab87ac7-b833-4202-8f81-b5e357041a23 None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:34.707677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ab87ac7-b833-4202-8f81-b5e357041a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:34.713759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:34.812700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 677/1359] 10.222.0.32 () {58 vars in 1058 bytes} [Mon May 30 11:56:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 11:56:34.820587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdd32e-f1de-4bf3-b11b-6ad8b8166881 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:34.836993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdd32e-f1de-4bf3-b11b-6ad8b8166881 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:34.843735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7fae5-3d19-440f-b1bc-f3d89031b002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:34.844422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7fae5-3d19-440f-b1bc-f3d89031b002 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:34.844740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7fae5-3d19-440f-b1bc-f3d89031b002 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:34.844982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7fae5-3d19-440f-b1bc-f3d89031b002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:34.854849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7fae5-3d19-440f-b1bc-f3d89031b002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:34.856803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7fae5-3d19-440f-b1bc-f3d89031b002 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:34.867267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 683/1360] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:37.408611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9fd68-0a33-4fbf-a90a-6a4a8e32f680 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:37.409559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9fd68-0a33-4fbf-a90a-6a4a8e32f680 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:37.409951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9fd68-0a33-4fbf-a90a-6a4a8e32f680 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:37.411957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 678/1361] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:37.417759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858866-6ad2-4066-9a5a-49feb3960c6b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:37.418556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858866-6ad2-4066-9a5a-49feb3960c6b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:37.418848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858866-6ad2-4066-9a5a-49feb3960c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:37.438154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fd858866-6ad2-4066-9a5a-49feb3960c6b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:37.439504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fd858866-6ad2-4066-9a5a-49feb3960c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:37.447851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 684/1362] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:37.492506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9fd68-0a33-4fbf-a90a-6a4a8e32f680 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:37.507400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9fd68-0a33-4fbf-a90a-6a4a8e32f680 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:37.513716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153ec42-5033-4005-b1a4-b7427e867890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:37.514359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ec42-5033-4005-b1a4-b7427e867890 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:37.514638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ec42-5033-4005-b1a4-b7427e867890 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:37.514913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ec42-5033-4005-b1a4-b7427e867890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:37.524016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153ec42-5033-4005-b1a4-b7427e867890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:37.526276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153ec42-5033-4005-b1a4-b7427e867890 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:37.536381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 679/1363] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:40.224522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1c6d7-a32b-4eff-ac35-91f3727105ff None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:40.225100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1c6d7-a32b-4eff-ac35-91f3727105ff None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:40.225100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1c6d7-a32b-4eff-ac35-91f3727105ff None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:40.228577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 685/1364] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:40.233483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44368d-2ae1-4f3c-85d7-e4bf645c3bbc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:40.233936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44368d-2ae1-4f3c-85d7-e4bf645c3bbc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:40.234029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44368d-2ae1-4f3c-85d7-e4bf645c3bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:40.256962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6e44368d-2ae1-4f3c-85d7-e4bf645c3bbc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:40.258708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6e44368d-2ae1-4f3c-85d7-e4bf645c3bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:40.267619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 680/1365] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:40.317372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b1769-d6f5-4365-84d0-9e775e486947 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:40.317658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b1769-d6f5-4365-84d0-9e775e486947 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:40.318288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b1769-d6f5-4365-84d0-9e775e486947 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:40.320121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 686/1366] 10.222.0.32 () {54 vars in 961 bytes} [Mon May 30 11:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:40.326573 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0ba60-5292-4b42-b713-2018226292d5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:40.326961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0ba60-5292-4b42-b713-2018226292d5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:40.327314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0ba60-5292-4b42-b713-2018226292d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:40.346261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-baa0ba60-5292-4b42-b713-2018226292d5 None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:40.347367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-baa0ba60-5292-4b42-b713-2018226292d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:40.357382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 681/1367] 10.222.0.32 () {58 vars in 1058 bytes} [Mon May 30 11:56:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:40.365139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b1769-d6f5-4365-84d0-9e775e486947 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:40.380465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b1769-d6f5-4365-84d0-9e775e486947 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:40.386751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b96de-e190-44b4-b7e6-90e38eeabb5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:40.387390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b96de-e190-44b4-b7e6-90e38eeabb5f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:40.387676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b96de-e190-44b4-b7e6-90e38eeabb5f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:40.387956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b96de-e190-44b4-b7e6-90e38eeabb5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:40.396558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b96de-e190-44b4-b7e6-90e38eeabb5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:40.398141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b96de-e190-44b4-b7e6-90e38eeabb5f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:40.407550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 687/1368] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:42.768354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c1dce-558e-4c05-bae1-0e5775516875 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:42.768919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c1dce-558e-4c05-bae1-0e5775516875 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:42.768919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c1dce-558e-4c05-bae1-0e5775516875 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:42.770476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 682/1369] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:42.776696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372fad5-e014-4823-a50e-3a916727ca9a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:42.777247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372fad5-e014-4823-a50e-3a916727ca9a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:42.777247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372fad5-e014-4823-a50e-3a916727ca9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:42.796773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3372fad5-e014-4823-a50e-3a916727ca9a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:42.797967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3372fad5-e014-4823-a50e-3a916727ca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:42.806214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 688/1370] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:42.852225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c1dce-558e-4c05-bae1-0e5775516875 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:42.867781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c1dce-558e-4c05-bae1-0e5775516875 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:42.874061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954fdf9-b18d-4678-8457-5db213854249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:42.874815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954fdf9-b18d-4678-8457-5db213854249 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:42.875170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954fdf9-b18d-4678-8457-5db213854249 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:42.875617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954fdf9-b18d-4678-8457-5db213854249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:42.885394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954fdf9-b18d-4678-8457-5db213854249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:42.887333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954fdf9-b18d-4678-8457-5db213854249 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:42.898915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 683/1371] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:43.089736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187e80d-834d-45d8-9e68-86c1f4e6db50 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:43.090128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187e80d-834d-45d8-9e68-86c1f4e6db50 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:43.090307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187e80d-834d-45d8-9e68-86c1f4e6db50 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:43.092016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 689/1372] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:43.098112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b4930-0c70-4adf-bd71-6638dac43310 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:43.098406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b4930-0c70-4adf-bd71-6638dac43310 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:43.098871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b4930-0c70-4adf-bd71-6638dac43310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:43.118953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f97b4930-0c70-4adf-bd71-6638dac43310 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:43.120884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f97b4930-0c70-4adf-bd71-6638dac43310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:43.129420 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 684/1373] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:43.405627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474156d7-84da-4def-8519-579bfc293641 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:43.406034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474156d7-84da-4def-8519-579bfc293641 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:43.406406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474156d7-84da-4def-8519-579bfc293641 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:43.407856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 690/1374] 10.222.0.32 () {54 vars in 963 bytes} [Mon May 30 11:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:43.413020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa721-07a8-41a6-842c-3c544348b799 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:43.413317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa721-07a8-41a6-842c-3c544348b799 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:43.413603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa721-07a8-41a6-842c-3c544348b799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:43.470327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-36caa721-07a8-41a6-842c-3c544348b799 None None] MFA Rules not processed for user `7c3c27e908e34977b248042f96c70dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:43.471549 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-36caa721-07a8-41a6-842c-3c544348b799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:43.536104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 685/1375] 10.222.0.32 () {58 vars in 1060 bytes} [Mon May 30 11:56:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:43.545294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474156d7-84da-4def-8519-579bfc293641 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:43.556417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474156d7-84da-4def-8519-579bfc293641 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:43.561700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23111b7-98eb-4c4f-8f6e-bd635e422035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:43.562635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23111b7-98eb-4c4f-8f6e-bd635e422035 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:43.563088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23111b7-98eb-4c4f-8f6e-bd635e422035 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:43.563522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23111b7-98eb-4c4f-8f6e-bd635e422035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:43.577836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23111b7-98eb-4c4f-8f6e-bd635e422035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:43.579656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23111b7-98eb-4c4f-8f6e-bd635e422035 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:43.595983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 691/1376] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:45.250973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b023fd-2b3c-4f55-92f8-d3780d6111ca None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:45.251496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b023fd-2b3c-4f55-92f8-d3780d6111ca None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:45.251592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b023fd-2b3c-4f55-92f8-d3780d6111ca None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:45.252931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 686/1377] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:45.258203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f3e3f-aaed-4e7c-8c0c-7b76023e80f7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:45.258641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f3e3f-aaed-4e7c-8c0c-7b76023e80f7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:45.258801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f3e3f-aaed-4e7c-8c0c-7b76023e80f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:45.278077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-369f3e3f-aaed-4e7c-8c0c-7b76023e80f7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:45.279290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-369f3e3f-aaed-4e7c-8c0c-7b76023e80f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:45.287804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 692/1378] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:45.332862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b023fd-2b3c-4f55-92f8-d3780d6111ca None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:45.349861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b023fd-2b3c-4f55-92f8-d3780d6111ca None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:45.356193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b799041-6474-4516-95e9-e0a950b6a5ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:45.356876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b799041-6474-4516-95e9-e0a950b6a5ff service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:45.357162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b799041-6474-4516-95e9-e0a950b6a5ff service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:45.357438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b799041-6474-4516-95e9-e0a950b6a5ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:45.367239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b799041-6474-4516-95e9-e0a950b6a5ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:45.368940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b799041-6474-4516-95e9-e0a950b6a5ff service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:45.380941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 687/1379] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:47.726616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dc39b-2144-47e4-aac3-71f09b8eef76 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:47.727155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dc39b-2144-47e4-aac3-71f09b8eef76 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:47.727155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dc39b-2144-47e4-aac3-71f09b8eef76 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:47.728553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 693/1380] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:47.734469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f3008-b6c5-4d68-a052-20a3150139e2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:47.734946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f3008-b6c5-4d68-a052-20a3150139e2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:47.735062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f3008-b6c5-4d68-a052-20a3150139e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:47.743113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:47.757865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-999f3008-b6c5-4d68-a052-20a3150139e2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:47.760999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-999f3008-b6c5-4d68-a052-20a3150139e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:47.773649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 688/1381] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:47.818374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980dc39b-2144-47e4-aac3-71f09b8eef76 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:47.834184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980dc39b-2144-47e4-aac3-71f09b8eef76 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:47.840252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9fc02-7b4d-49df-9a5a-b5ad2c89af8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:47.840996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9fc02-7b4d-49df-9a5a-b5ad2c89af8d service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:47.841363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9fc02-7b4d-49df-9a5a-b5ad2c89af8d service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:47.841720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9fc02-7b4d-49df-9a5a-b5ad2c89af8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:47.850979 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9fc02-7b4d-49df-9a5a-b5ad2c89af8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:47.852754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9fc02-7b4d-49df-9a5a-b5ad2c89af8d service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:47.863291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 694/1382] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:50.209278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731b1d2-fabd-47f7-84c9-11195f8c90e7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:50.209807 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731b1d2-fabd-47f7-84c9-11195f8c90e7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:50.209807 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731b1d2-fabd-47f7-84c9-11195f8c90e7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:50.211179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 689/1383] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:50.217269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc55f8-f167-4d8c-8068-7f60a9d8f109 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:50.217737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc55f8-f167-4d8c-8068-7f60a9d8f109 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:50.217833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc55f8-f167-4d8c-8068-7f60a9d8f109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:50.236675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-efcc55f8-f167-4d8c-8068-7f60a9d8f109 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:50.237871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-efcc55f8-f167-4d8c-8068-7f60a9d8f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:50.247174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 695/1384] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 11:56:50.289938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731b1d2-fabd-47f7-84c9-11195f8c90e7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:50.310129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731b1d2-fabd-47f7-84c9-11195f8c90e7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:50.317162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b0c09-933e-4546-8e90-283d32972e76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:50.317847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0c09-933e-4546-8e90-283d32972e76 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:50.318134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0c09-933e-4546-8e90-283d32972e76 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:50.318533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0c09-933e-4546-8e90-283d32972e76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:50.330285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b0c09-933e-4546-8e90-283d32972e76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:50.331896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b0c09-933e-4546-8e90-283d32972e76 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:50.345604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 690/1385] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:52.842410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d8a22-aaaa-4ed9-b6ff-d1fb4b2e456b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:52.842921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d8a22-aaaa-4ed9-b6ff-d1fb4b2e456b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:52.842921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d8a22-aaaa-4ed9-b6ff-d1fb4b2e456b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:52.844357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 696/1386] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:56:52.849894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d77a2-fea1-45fa-9ecb-139e8c456fdf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:52.850287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d77a2-fea1-45fa-9ecb-139e8c456fdf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:52.850456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d77a2-fea1-45fa-9ecb-139e8c456fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:52.870242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-370d77a2-fea1-45fa-9ecb-139e8c456fdf None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:52.871502 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-370d77a2-fea1-45fa-9ecb-139e8c456fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:52.881206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 691/1387] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:52.930679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d8a22-aaaa-4ed9-b6ff-d1fb4b2e456b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:52.945875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d8a22-aaaa-4ed9-b6ff-d1fb4b2e456b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:52.951488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb1b10-34fa-43d7-8c26-8b9fbab3a760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:52.952174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb1b10-34fa-43d7-8c26-8b9fbab3a760 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:52.952462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb1b10-34fa-43d7-8c26-8b9fbab3a760 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:52.952750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb1b10-34fa-43d7-8c26-8b9fbab3a760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:52.961558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb1b10-34fa-43d7-8c26-8b9fbab3a760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:52.963667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb1b10-34fa-43d7-8c26-8b9fbab3a760 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:52.973703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 697/1388] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:55.451958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00e942-4d0c-4aff-bcae-6c093c302d94 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:55.452703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00e942-4d0c-4aff-bcae-6c093c302d94 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:55.452703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00e942-4d0c-4aff-bcae-6c093c302d94 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:55.453928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 692/1389] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:55.459680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909be78-a984-46ce-869d-740f36c710fb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:55.460187 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909be78-a984-46ce-869d-740f36c710fb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:55.460287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909be78-a984-46ce-869d-740f36c710fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:55.478656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d909be78-a984-46ce-869d-740f36c710fb None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:55.479851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d909be78-a984-46ce-869d-740f36c710fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:55.487934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 698/1390] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:55.538240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00e942-4d0c-4aff-bcae-6c093c302d94 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:55.553571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00e942-4d0c-4aff-bcae-6c093c302d94 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:55.559377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9cd3b-f78e-4ba5-8dd9-35451166d979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:55.560038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9cd3b-f78e-4ba5-8dd9-35451166d979 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:55.560424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9cd3b-f78e-4ba5-8dd9-35451166d979 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:55.560715 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9cd3b-f78e-4ba5-8dd9-35451166d979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:55.569346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9cd3b-f78e-4ba5-8dd9-35451166d979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:55.570995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9cd3b-f78e-4ba5-8dd9-35451166d979 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:55.582340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 693/1391] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:57.958881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11673af-8377-4500-af2c-e1cf48153d36 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:57.959428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11673af-8377-4500-af2c-e1cf48153d36 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:57.959428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11673af-8377-4500-af2c-e1cf48153d36 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:57.960827 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 699/1392] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:57.967604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c02d2-636a-43aa-b27c-4d4b8b6b0e01 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:57.968087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c02d2-636a-43aa-b27c-4d4b8b6b0e01 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:57.968211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c02d2-636a-43aa-b27c-4d4b8b6b0e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:57.987558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-850c02d2-636a-43aa-b27c-4d4b8b6b0e01 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:57.988808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-850c02d2-636a-43aa-b27c-4d4b8b6b0e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:57.996940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 694/1393] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:58.042437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11673af-8377-4500-af2c-e1cf48153d36 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:58.058027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11673af-8377-4500-af2c-e1cf48153d36 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:58.063521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b7288-50b6-47a0-80d8-1b30725ad52f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:58.064174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b7288-50b6-47a0-80d8-1b30725ad52f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:58.064489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b7288-50b6-47a0-80d8-1b30725ad52f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:58.064777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b7288-50b6-47a0-80d8-1b30725ad52f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:58.074196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b7288-50b6-47a0-80d8-1b30725ad52f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:58.075816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b7288-50b6-47a0-80d8-1b30725ad52f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:58.085577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 700/1394] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:56:59.692964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c76f9c-e745-4bb5-9a48-4b70146bae14 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:59.693536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c76f9c-e745-4bb5-9a48-4b70146bae14 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:59.693536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c76f9c-e745-4bb5-9a48-4b70146bae14 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:59.694933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 695/1395] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:56:59.700605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093463de-7796-4998-9e00-cdfe8cfa788f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:59.701075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093463de-7796-4998-9e00-cdfe8cfa788f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:59.701183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093463de-7796-4998-9e00-cdfe8cfa788f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:59.707278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=52 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:56:59.721438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-093463de-7796-4998-9e00-cdfe8cfa788f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:59.722643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-093463de-7796-4998-9e00-cdfe8cfa788f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:59.731393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 701/1396] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:56:59.788645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce8505-a65b-4467-9a33-d0de25c49613 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:59.788939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce8505-a65b-4467-9a33-d0de25c49613 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:59.789217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce8505-a65b-4467-9a33-d0de25c49613 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:59.790565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 696/1397] 10.222.0.32 () {54 vars in 962 bytes} [Mon May 30 11:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:56:59.796772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636185b-85f5-46ff-bcb9-b8d87807a3cd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:59.797156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636185b-85f5-46ff-bcb9-b8d87807a3cd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:59.797512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636185b-85f5-46ff-bcb9-b8d87807a3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:59.848241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b636185b-85f5-46ff-bcb9-b8d87807a3cd None None] MFA Rules not processed for user `ad37bdce39f64836b0cd14613cabe91f`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:56:59.849366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b636185b-85f5-46ff-bcb9-b8d87807a3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:56:59.915677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 702/1398] 10.222.0.32 () {58 vars in 1059 bytes} [Mon May 30 11:56:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 11:56:59.923998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce8505-a65b-4467-9a33-d0de25c49613 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:56:59.935457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce8505-a65b-4467-9a33-d0de25c49613 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:56:59.939506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedaa66-3108-411a-8bbb-765d2d531041 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad37bdce39f64836b0cd14613cabe91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:56:59.940247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedaa66-3108-411a-8bbb-765d2d531041 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:56:59.940595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedaa66-3108-411a-8bbb-765d2d531041 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:56:59.940943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedaa66-3108-411a-8bbb-765d2d531041 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:56:59.951325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedaa66-3108-411a-8bbb-765d2d531041 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:56:59.952871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedaa66-3108-411a-8bbb-765d2d531041 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:56:59.963187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 697/1399] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:00.429604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf5d8-f4fa-4df2-bd46-94dae21a8104 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:00.429905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf5d8-f4fa-4df2-bd46-94dae21a8104 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:00.430189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf5d8-f4fa-4df2-bd46-94dae21a8104 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:00.432192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 703/1400] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:00.437655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d0915-bff9-49b2-847d-0d52bc5cb402 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:00.437949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d0915-bff9-49b2-847d-0d52bc5cb402 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:00.438229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d0915-bff9-49b2-847d-0d52bc5cb402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:00.457251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-150d0915-bff9-49b2-847d-0d52bc5cb402 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:00.458458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-150d0915-bff9-49b2-847d-0d52bc5cb402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:00.466483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 698/1401] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 11:57:00.522632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acf5d8-f4fa-4df2-bd46-94dae21a8104 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:00.526585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3247ea-103c-4204-b431-632cc90e3a8b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:00.526872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3247ea-103c-4204-b431-632cc90e3a8b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:00.527152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3247ea-103c-4204-b431-632cc90e3a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:00.543106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acf5d8-f4fa-4df2-bd46-94dae21a8104 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:00.548773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907276d-a739-41b2-a46d-f9288d8f188c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:00.549492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907276d-a739-41b2-a46d-f9288d8f188c service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:00.549850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907276d-a739-41b2-a46d-f9288d8f188c service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:00.550228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907276d-a739-41b2-a46d-f9288d8f188c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:00.559914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907276d-a739-41b2-a46d-f9288d8f188c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:00.561696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907276d-a739-41b2-a46d-f9288d8f188c service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:00.575621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 704/1402] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:00.578818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7e3247ea-103c-4204-b431-632cc90e3a8b None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:00.580340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7e3247ea-103c-4204-b431-632cc90e3a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:00.657238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 699/1403] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 11:57:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 11:57:00.706445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512d3e-5867-4307-a7fc-a3f9063fad92 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:00.707042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512d3e-5867-4307-a7fc-a3f9063fad92 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:00.707042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33512d3e-5867-4307-a7fc-a3f9063fad92 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:00.708530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 705/1404] 10.222.0.32 () {54 vars in 942 bytes} [Mon May 30 11:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:00.716714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512023-9128-4561-90a6-4de669bfa524 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:00.717239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512023-9128-4561-90a6-4de669bfa524 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:00.717239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9512023-9128-4561-90a6-4de669bfa524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:00.775721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b9512023-9128-4561-90a6-4de669bfa524 None None] MFA Rules not processed for user `e0cb45c1eb6040548634022673a51e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:00.777186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b9512023-9128-4561-90a6-4de669bfa524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:00.824523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:57:00.858449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 700/1405] 10.222.0.32 () {58 vars in 1039 bytes} [Mon May 30 11:57:00 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:00.867632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33512d3e-5867-4307-a7fc-a3f9063fad92 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:00.882289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33512d3e-5867-4307-a7fc-a3f9063fad92 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:00.887738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c576a8f-5526-4a4b-82ad-0ae4b7f4ad98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:00.888425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576a8f-5526-4a4b-82ad-0ae4b7f4ad98 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:00.888730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576a8f-5526-4a4b-82ad-0ae4b7f4ad98 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:00.889012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c576a8f-5526-4a4b-82ad-0ae4b7f4ad98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:00.898119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c576a8f-5526-4a4b-82ad-0ae4b7f4ad98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:00.899746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c576a8f-5526-4a4b-82ad-0ae4b7f4ad98 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:00.908214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 706/1406] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 11:57:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 11:57:02.945492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf4f73-62df-4403-9b32-a006131eeb23 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:02.946034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf4f73-62df-4403-9b32-a006131eeb23 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:02.946034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf4f73-62df-4403-9b32-a006131eeb23 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:02.947891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 701/1407] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:02.953942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bee34-b42a-4e89-b442-7be1162ef8da None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:02.954432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bee34-b42a-4e89-b442-7be1162ef8da None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:02.954540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bee34-b42a-4e89-b442-7be1162ef8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:02.973667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ec0bee34-b42a-4e89-b442-7be1162ef8da None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:02.974876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ec0bee34-b42a-4e89-b442-7be1162ef8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:02.985070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 707/1408] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:03.035260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf4f73-62df-4403-9b32-a006131eeb23 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:03.050056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf4f73-62df-4403-9b32-a006131eeb23 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:03.059189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfc846-c434-4270-bd11-532b39a001da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:03.059846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfc846-c434-4270-bd11-532b39a001da service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:03.060128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfc846-c434-4270-bd11-532b39a001da service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:03.060444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfc846-c434-4270-bd11-532b39a001da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:03.069452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfc846-c434-4270-bd11-532b39a001da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:03.071051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfc846-c434-4270-bd11-532b39a001da service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:03.081412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 702/1409] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:12.968312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=55 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:57:13.077769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=18 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:57:24.508551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3fa0a-1474-4dc3-add0-a6294bf3ab4f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:24.509606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3fa0a-1474-4dc3-add0-a6294bf3ab4f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:24.509606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3fa0a-1474-4dc3-add0-a6294bf3ab4f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:24.511170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 708/1410] 10.222.0.32 () {54 vars in 918 bytes} [Mon May 30 11:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:24.517866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef6600-a8c0-4ec7-8031-b514a1a22da3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:24.518375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef6600-a8c0-4ec7-8031-b514a1a22da3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:24.518494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef6600-a8c0-4ec7-8031-b514a1a22da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:24.537022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ceef6600-a8c0-4ec7-8031-b514a1a22da3 None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:24.538225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ceef6600-a8c0-4ec7-8031-b514a1a22da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:24.547396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 703/1411] 10.222.0.32 () {58 vars in 1015 bytes} [Mon May 30 11:57:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:24.563486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60559e09-5a48-4c3e-9625-81fac9842a6a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:24.563825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60559e09-5a48-4c3e-9625-81fac9842a6a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:24.564141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60559e09-5a48-4c3e-9625-81fac9842a6a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:24.565538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 709/1412] 10.222.0.32 () {54 vars in 918 bytes} [Mon May 30 11:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:24.570433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41ad3-7e7c-4a59-a861-0c0ceb8ef9f1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:24.570728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41ad3-7e7c-4a59-a861-0c0ceb8ef9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:24.571001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a41ad3-7e7c-4a59-a861-0c0ceb8ef9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:24.590932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-70a41ad3-7e7c-4a59-a861-0c0ceb8ef9f1 None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:24.592123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-70a41ad3-7e7c-4a59-a861-0c0ceb8ef9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:24.601215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 704/1413] 10.222.0.32 () {58 vars in 1015 bytes} [Mon May 30 11:57:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:27.381829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa423b-cc29-4363-a8a8-76094b4662a2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:27.382376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa423b-cc29-4363-a8a8-76094b4662a2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:27.382376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa423b-cc29-4363-a8a8-76094b4662a2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:27.383747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:27.389214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33939d44-01ed-4138-b633-e86099c95ebd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:27.389650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33939d44-01ed-4138-b633-e86099c95ebd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:27.389837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33939d44-01ed-4138-b633-e86099c95ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:27.408680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-33939d44-01ed-4138-b633-e86099c95ebd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:27.409857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-33939d44-01ed-4138-b633-e86099c95ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:27.418447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:27.461230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa423b-cc29-4363-a8a8-76094b4662a2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:27.476412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa423b-cc29-4363-a8a8-76094b4662a2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:27.482447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61cea2-2be4-40bf-9ca0-a1654ccff6b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:27.483091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61cea2-2be4-40bf-9ca0-a1654ccff6b5 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:27.483381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61cea2-2be4-40bf-9ca0-a1654ccff6b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:27.483663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61cea2-2be4-40bf-9ca0-a1654ccff6b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:27.492086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61cea2-2be4-40bf-9ca0-a1654ccff6b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:27.493710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61cea2-2be4-40bf-9ca0-a1654ccff6b5 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:27.503321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 711/1416] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:28.497264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d4a44-4cf1-4fc8-94d9-2a9e21f3dd0f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:28.497840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d4a44-4cf1-4fc8-94d9-2a9e21f3dd0f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:28.497840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d4a44-4cf1-4fc8-94d9-2a9e21f3dd0f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:28.499205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 11:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:28.504232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56f424-563d-4e3b-8803-0b9f6f1b7d86 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:28.504647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56f424-563d-4e3b-8803-0b9f6f1b7d86 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:28.504808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56f424-563d-4e3b-8803-0b9f6f1b7d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:28.523654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0f56f424-563d-4e3b-8803-0b9f6f1b7d86 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:28.525544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0f56f424-563d-4e3b-8803-0b9f6f1b7d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:28.533421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 11:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:28.693530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d4a44-4cf1-4fc8-94d9-2a9e21f3dd0f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:28.708235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d4a44-4cf1-4fc8-94d9-2a9e21f3dd0f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:28.714546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249c3a-b743-4240-99f0-8a0f569659fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:28.715195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249c3a-b743-4240-99f0-8a0f569659fe service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:28.715477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249c3a-b743-4240-99f0-8a0f569659fe service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:28.715750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249c3a-b743-4240-99f0-8a0f569659fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:28.724371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249c3a-b743-4240-99f0-8a0f569659fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:28.725947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249c3a-b743-4240-99f0-8a0f569659fe service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:28.735628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 707/1419] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:29.677091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bb391-3d47-4458-acd2-45634d3fe888 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:29.677677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bb391-3d47-4458-acd2-45634d3fe888 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:29.677677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bb391-3d47-4458-acd2-45634d3fe888 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:29.679057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 11:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 11:57:29.684421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5d83-1488-4b3b-8625-b1f86ccffdfe None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:29.684873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5d83-1488-4b3b-8625-b1f86ccffdfe None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:29.684983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5d83-1488-4b3b-8625-b1f86ccffdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:29.702827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-69db5d83-1488-4b3b-8625-b1f86ccffdfe None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:29.704008 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-69db5d83-1488-4b3b-8625-b1f86ccffdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:29.711637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:29.863796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bb391-3d47-4458-acd2-45634d3fe888 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:29.879015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bb391-3d47-4458-acd2-45634d3fe888 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:29.888495 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40178ef-e26d-4d88-8b5c-2dca8683a750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:29.889539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40178ef-e26d-4d88-8b5c-2dca8683a750 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:29.889828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40178ef-e26d-4d88-8b5c-2dca8683a750 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:29.890106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40178ef-e26d-4d88-8b5c-2dca8683a750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:29.900944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40178ef-e26d-4d88-8b5c-2dca8683a750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:29.902727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40178ef-e26d-4d88-8b5c-2dca8683a750 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:29.912133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 714/1422] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:30.842694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec741e8-df65-4211-a57c-4ef6329dff54 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:30.843313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec741e8-df65-4211-a57c-4ef6329dff54 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:30.843313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec741e8-df65-4211-a57c-4ef6329dff54 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:30.844587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 11:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:30.849750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad622b-1157-4745-bbf9-1d33208e2c39 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:30.850150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad622b-1157-4745-bbf9-1d33208e2c39 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:30.850320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad622b-1157-4745-bbf9-1d33208e2c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:30.869453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-24ad622b-1157-4745-bbf9-1d33208e2c39 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:30.870832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-24ad622b-1157-4745-bbf9-1d33208e2c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:30.878790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:31.029188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec741e8-df65-4211-a57c-4ef6329dff54 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:31.044248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec741e8-df65-4211-a57c-4ef6329dff54 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:31.049660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d6460-b40a-4e79-ac53-cffa7871137b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:31.050300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d6460-b40a-4e79-ac53-cffa7871137b service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:31.050587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d6460-b40a-4e79-ac53-cffa7871137b service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:31.050868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d6460-b40a-4e79-ac53-cffa7871137b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:31.059847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d6460-b40a-4e79-ac53-cffa7871137b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:31.061473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d6460-b40a-4e79-ac53-cffa7871137b service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:31.070908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 710/1425] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:31.991848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a58a9e-0c2d-49a6-9192-37af0d36fe3e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:31.992441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a58a9e-0c2d-49a6-9192-37af0d36fe3e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:31.992441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a58a9e-0c2d-49a6-9192-37af0d36fe3e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:31.993939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 11:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:32.000651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc5bf-f09c-42d6-aca7-4c3d6a8904a9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:32.001189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc5bf-f09c-42d6-aca7-4c3d6a8904a9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:32.001609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc5bf-f09c-42d6-aca7-4c3d6a8904a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:32.022191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c5fdc5bf-f09c-42d6-aca7-4c3d6a8904a9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:32.023963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c5fdc5bf-f09c-42d6-aca7-4c3d6a8904a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:32.036116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 11:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:32.187032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a58a9e-0c2d-49a6-9192-37af0d36fe3e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:32.201816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a58a9e-0c2d-49a6-9192-37af0d36fe3e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:32.207163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85bf7e-5823-4806-8eef-d2c652539283 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:32.207799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85bf7e-5823-4806-8eef-d2c652539283 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:32.208086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85bf7e-5823-4806-8eef-d2c652539283 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:32.208404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85bf7e-5823-4806-8eef-d2c652539283 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:32.217220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85bf7e-5823-4806-8eef-d2c652539283 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:32.218808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85bf7e-5823-4806-8eef-d2c652539283 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:32.228085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 717/1428] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:33.095833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815df23f-75cc-49b1-8073-309b3ed3fe49 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:33.096433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815df23f-75cc-49b1-8073-309b3ed3fe49 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:33.096433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815df23f-75cc-49b1-8073-309b3ed3fe49 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:33.097796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 11:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:33.102800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e19ea-e98b-4ff3-90b6-164651f7637b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:33.103796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e19ea-e98b-4ff3-90b6-164651f7637b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:33.104080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e19ea-e98b-4ff3-90b6-164651f7637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:33.123020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0f7e19ea-e98b-4ff3-90b6-164651f7637b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:33.124277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0f7e19ea-e98b-4ff3-90b6-164651f7637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:33.132051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 11:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 11:57:33.287849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815df23f-75cc-49b1-8073-309b3ed3fe49 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:33.303588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815df23f-75cc-49b1-8073-309b3ed3fe49 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:33.309940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14daa0-0f50-421d-98c2-5bf2488dcc95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:33.310578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14daa0-0f50-421d-98c2-5bf2488dcc95 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:33.310860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14daa0-0f50-421d-98c2-5bf2488dcc95 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:33.311136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14daa0-0f50-421d-98c2-5bf2488dcc95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:33.319342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14daa0-0f50-421d-98c2-5bf2488dcc95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:33.321068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14daa0-0f50-421d-98c2-5bf2488dcc95 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:33.331656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 713/1431] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:34.201127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad6993-e543-403a-b2f3-e50ab1b4dc6f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:34.201680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad6993-e543-403a-b2f3-e50ab1b4dc6f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:34.201680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad6993-e543-403a-b2f3-e50ab1b4dc6f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:34.203136 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 11:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:34.208286 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f042d-5ccc-4d4e-bff8-6428568387fa None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:34.208700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f042d-5ccc-4d4e-bff8-6428568387fa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:34.208857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f042d-5ccc-4d4e-bff8-6428568387fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:34.227133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-372f042d-5ccc-4d4e-bff8-6428568387fa None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:34.229034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-372f042d-5ccc-4d4e-bff8-6428568387fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:34.237832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 11:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:34.391910 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad6993-e543-403a-b2f3-e50ab1b4dc6f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:34.409326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad6993-e543-403a-b2f3-e50ab1b4dc6f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:34.417631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb292081-0274-4780-9218-981a6a6127da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:34.418284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292081-0274-4780-9218-981a6a6127da service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:34.418573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292081-0274-4780-9218-981a6a6127da service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:34.418854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb292081-0274-4780-9218-981a6a6127da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:34.427445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb292081-0274-4780-9218-981a6a6127da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:34.429079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb292081-0274-4780-9218-981a6a6127da service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:34.438519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 720/1434] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:35.334929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0094-2aca-4579-a7a1-2d13e74b362a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:35.335501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0094-2aca-4579-a7a1-2d13e74b362a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:35.335501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0094-2aca-4579-a7a1-2d13e74b362a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:35.336885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 11:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:35.342621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494013b0-1d3f-44c3-9ea8-b25060e8a202 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:35.343074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494013b0-1d3f-44c3-9ea8-b25060e8a202 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:35.343192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494013b0-1d3f-44c3-9ea8-b25060e8a202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:35.361813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-494013b0-1d3f-44c3-9ea8-b25060e8a202 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:35.363036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-494013b0-1d3f-44c3-9ea8-b25060e8a202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:35.371320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 11:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:35.533442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0094-2aca-4579-a7a1-2d13e74b362a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:35.548349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0094-2aca-4579-a7a1-2d13e74b362a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:35.553795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0943c-1aae-4ac9-86f9-8f8f30cbf3ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:35.554453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0943c-1aae-4ac9-86f9-8f8f30cbf3ac service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:35.554736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0943c-1aae-4ac9-86f9-8f8f30cbf3ac service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:35.555019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0943c-1aae-4ac9-86f9-8f8f30cbf3ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:35.564051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0943c-1aae-4ac9-86f9-8f8f30cbf3ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:35.565681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0943c-1aae-4ac9-86f9-8f8f30cbf3ac service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:35.575870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 716/1437] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:36.455687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea680c-9434-4289-ad78-4e772c259edb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:36.456294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea680c-9434-4289-ad78-4e772c259edb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:36.456294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea680c-9434-4289-ad78-4e772c259edb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:36.457694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 11:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:36.463723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af690af-4e0b-4bf4-8876-79d2fe239c18 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:36.464237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af690af-4e0b-4bf4-8876-79d2fe239c18 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:36.464449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af690af-4e0b-4bf4-8876-79d2fe239c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:36.483407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0af690af-4e0b-4bf4-8876-79d2fe239c18 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:36.484663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0af690af-4e0b-4bf4-8876-79d2fe239c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:36.493784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 11:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:36.645158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea680c-9434-4289-ad78-4e772c259edb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:36.660553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea680c-9434-4289-ad78-4e772c259edb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:36.666604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f66b7-159d-4ca4-8392-65f135007cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:36.667244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f66b7-159d-4ca4-8392-65f135007cfb service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:36.667532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f66b7-159d-4ca4-8392-65f135007cfb service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:36.667813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f66b7-159d-4ca4-8392-65f135007cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:36.676285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f66b7-159d-4ca4-8392-65f135007cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:36.677897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f66b7-159d-4ca4-8392-65f135007cfb service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:36.687611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 723/1440] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:38.842091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3364-c170-449a-b809-2019683bf3bd None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:38.842599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3364-c170-449a-b809-2019683bf3bd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:38.842599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3364-c170-449a-b809-2019683bf3bd None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:38.843937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:38.849633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cc61d-dc72-4666-b135-87dd12baaf61 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:38.850035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cc61d-dc72-4666-b135-87dd12baaf61 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:38.850190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cc61d-dc72-4666-b135-87dd12baaf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:38.868708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-451cc61d-dc72-4666-b135-87dd12baaf61 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:38.870322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-451cc61d-dc72-4666-b135-87dd12baaf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:38.878445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:38.917668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d3364-c170-449a-b809-2019683bf3bd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:38.928380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d3364-c170-449a-b809-2019683bf3bd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:38.931969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b4cb7-e477-4ace-9594-f72f73b577a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:38.932644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4cb7-e477-4ace-9594-f72f73b577a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:38.932934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4cb7-e477-4ace-9594-f72f73b577a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:38.933212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4cb7-e477-4ace-9594-f72f73b577a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:38.942050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4cb7-e477-4ace-9594-f72f73b577a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:38.943474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b4cb7-e477-4ace-9594-f72f73b577a5 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:38.953409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 719/1443] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:41.381606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22425232-eace-4b59-8268-88877d0cc056 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:41.382175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22425232-eace-4b59-8268-88877d0cc056 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:41.382175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22425232-eace-4b59-8268-88877d0cc056 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:41.383523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:41.388967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a6896-a90a-4a8d-ab9d-a0ee00a96edd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:41.389421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a6896-a90a-4a8d-ab9d-a0ee00a96edd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:41.389520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a6896-a90a-4a8d-ab9d-a0ee00a96edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:41.409739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-470a6896-a90a-4a8d-ab9d-a0ee00a96edd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:41.411657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-470a6896-a90a-4a8d-ab9d-a0ee00a96edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:41.422208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:41.460478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22425232-eace-4b59-8268-88877d0cc056 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:41.471195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22425232-eace-4b59-8268-88877d0cc056 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:41.474712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa702de-b276-446d-8e8b-9d1ca4bcfe44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:41.475351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa702de-b276-446d-8e8b-9d1ca4bcfe44 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:41.475638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa702de-b276-446d-8e8b-9d1ca4bcfe44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:41.475913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa702de-b276-446d-8e8b-9d1ca4bcfe44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:41.485978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa702de-b276-446d-8e8b-9d1ca4bcfe44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:41.487483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa702de-b276-446d-8e8b-9d1ca4bcfe44 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:41.497667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 726/1446] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:44.074197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43c3c7-c9ba-4e63-aaac-aabf8a710909 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:44.074806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43c3c7-c9ba-4e63-aaac-aabf8a710909 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:44.074806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43c3c7-c9ba-4e63-aaac-aabf8a710909 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:44.076123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:44.081492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff3baa-9814-42bd-bcb5-25126c322d58 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:44.081912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff3baa-9814-42bd-bcb5-25126c322d58 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:44.082060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff3baa-9814-42bd-bcb5-25126c322d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:44.100593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ecff3baa-9814-42bd-bcb5-25126c322d58 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:44.101908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ecff3baa-9814-42bd-bcb5-25126c322d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:44.109760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:44.148584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43c3c7-c9ba-4e63-aaac-aabf8a710909 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:44.159620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43c3c7-c9ba-4e63-aaac-aabf8a710909 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:44.163251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83e89a-3975-4317-b3b5-168bf9940943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:44.163917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83e89a-3975-4317-b3b5-168bf9940943 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:44.164233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83e89a-3975-4317-b3b5-168bf9940943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:44.164521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83e89a-3975-4317-b3b5-168bf9940943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:44.174363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83e89a-3975-4317-b3b5-168bf9940943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:44.175793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83e89a-3975-4317-b3b5-168bf9940943 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:44.185885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 722/1449] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:46.661971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c46afd-f06a-4761-9ac6-ad2bfbd96b14 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:46.662529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c46afd-f06a-4761-9ac6-ad2bfbd96b14 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:46.662529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c46afd-f06a-4761-9ac6-ad2bfbd96b14 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:46.663908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:46.669773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54836385-5b1f-470b-b7be-1dca8c83d8e5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:46.670213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54836385-5b1f-470b-b7be-1dca8c83d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:46.670331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54836385-5b1f-470b-b7be-1dca8c83d8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:46.688605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-54836385-5b1f-470b-b7be-1dca8c83d8e5 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:46.689812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-54836385-5b1f-470b-b7be-1dca8c83d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:46.698594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:46.738063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c46afd-f06a-4761-9ac6-ad2bfbd96b14 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:46.749127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c46afd-f06a-4761-9ac6-ad2bfbd96b14 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:46.752752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3da5b-00ec-446a-a67d-9e53a3bab339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:46.753403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3da5b-00ec-446a-a67d-9e53a3bab339 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:46.753696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3da5b-00ec-446a-a67d-9e53a3bab339 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:46.753979 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3da5b-00ec-446a-a67d-9e53a3bab339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:46.763542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3da5b-00ec-446a-a67d-9e53a3bab339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:46.765016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3da5b-00ec-446a-a67d-9e53a3bab339 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:46.775325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 729/1452] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:49.111271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99b5a1-8cd5-48c4-87f5-e53472132064 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:49.111942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99b5a1-8cd5-48c4-87f5-e53472132064 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:49.111942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99b5a1-8cd5-48c4-87f5-e53472132064 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:49.113295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:49.119070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332a53-8e47-4c8c-a9de-3710392bc4d1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:49.119579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332a53-8e47-4c8c-a9de-3710392bc4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:49.119868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332a53-8e47-4c8c-a9de-3710392bc4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:49.138003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a9332a53-8e47-4c8c-a9de-3710392bc4d1 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:49.139189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a9332a53-8e47-4c8c-a9de-3710392bc4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:49.147155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:49.185425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99b5a1-8cd5-48c4-87f5-e53472132064 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:49.196001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99b5a1-8cd5-48c4-87f5-e53472132064 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:49.200076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89100e-7d99-4781-a78e-279c896688b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:49.200758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89100e-7d99-4781-a78e-279c896688b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:49.201043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89100e-7d99-4781-a78e-279c896688b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:49.201318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89100e-7d99-4781-a78e-279c896688b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:49.210118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89100e-7d99-4781-a78e-279c896688b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:49.211541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89100e-7d99-4781-a78e-279c896688b9 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:49.220857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 725/1455] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:51.567239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b2915-f756-4603-abb8-b42dfdd1a6db None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:51.567820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b2915-f756-4603-abb8-b42dfdd1a6db None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:51.567820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b2915-f756-4603-abb8-b42dfdd1a6db None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:51.569205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:51.575139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fda2b8-376f-4c01-abbb-fb826a014fba None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:51.575609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fda2b8-376f-4c01-abbb-fb826a014fba None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:51.575705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fda2b8-376f-4c01-abbb-fb826a014fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:51.593873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-53fda2b8-376f-4c01-abbb-fb826a014fba None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:51.595072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-53fda2b8-376f-4c01-abbb-fb826a014fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:51.603572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:51.641717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b2915-f756-4603-abb8-b42dfdd1a6db None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:51.643682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-699b2915-f756-4603-abb8-b42dfdd1a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:51.657716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b2915-f756-4603-abb8-b42dfdd1a6db None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:51.661693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61c0ee-591a-4643-9134-cd5c4979d956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:51.662339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61c0ee-591a-4643-9134-cd5c4979d956 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:51.662628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61c0ee-591a-4643-9134-cd5c4979d956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:51.662914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61c0ee-591a-4643-9134-cd5c4979d956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:51.673180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61c0ee-591a-4643-9134-cd5c4979d956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:51.675019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61c0ee-591a-4643-9134-cd5c4979d956 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:51.684961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 732/1458] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:54.032981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62a39-16d8-4c30-8198-716ab3686b29 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:54.033529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62a39-16d8-4c30-8198-716ab3686b29 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:54.033529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62a39-16d8-4c30-8198-716ab3686b29 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:54.034911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:54.040888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0d983-7b12-45bd-bc16-a31ff84fee1b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:54.041342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0d983-7b12-45bd-bc16-a31ff84fee1b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:54.041538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0d983-7b12-45bd-bc16-a31ff84fee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:54.047609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=70 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:57:54.061554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e4e0d983-7b12-45bd-bc16-a31ff84fee1b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:54.062811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e4e0d983-7b12-45bd-bc16-a31ff84fee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:54.071044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:54.108597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc62a39-16d8-4c30-8198-716ab3686b29 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:54.114053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=73 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:57:54.121091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc62a39-16d8-4c30-8198-716ab3686b29 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:54.124699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa601c-be5d-4fab-9b20-1cc5c93eaaf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:54.125333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa601c-be5d-4fab-9b20-1cc5c93eaaf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:54.125615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa601c-be5d-4fab-9b20-1cc5c93eaaf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:54.125891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa601c-be5d-4fab-9b20-1cc5c93eaaf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:54.134907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa601c-be5d-4fab-9b20-1cc5c93eaaf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:54.136362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa601c-be5d-4fab-9b20-1cc5c93eaaf4 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:54.145964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 728/1461] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:56.476677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b001d6e-a0b0-4e8a-9a6b-4eadbb927ec5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:56.477222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b001d6e-a0b0-4e8a-9a6b-4eadbb927ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:56.477356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b001d6e-a0b0-4e8a-9a6b-4eadbb927ec5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:56.478839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:56.484533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090d55b-74af-4e9c-9cf2-5188060a9203 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:56.484932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090d55b-74af-4e9c-9cf2-5188060a9203 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:56.485128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090d55b-74af-4e9c-9cf2-5188060a9203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:56.502879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7090d55b-74af-4e9c-9cf2-5188060a9203 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:56.504086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7090d55b-74af-4e9c-9cf2-5188060a9203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:56.512662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:56.550616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b001d6e-a0b0-4e8a-9a6b-4eadbb927ec5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:56.561773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b001d6e-a0b0-4e8a-9a6b-4eadbb927ec5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:56.565376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7d98b-8395-47b5-86d8-28d0c3418507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:56.566000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7d98b-8395-47b5-86d8-28d0c3418507 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:56.566289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7d98b-8395-47b5-86d8-28d0c3418507 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:56.566571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7d98b-8395-47b5-86d8-28d0c3418507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:56.577079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7d98b-8395-47b5-86d8-28d0c3418507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:56.578532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7d98b-8395-47b5-86d8-28d0c3418507 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:56.588572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 735/1464] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:57:58.959659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fde15-85ac-414d-b6bd-3fd4c760ee12 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:58.960228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fde15-85ac-414d-b6bd-3fd4c760ee12 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:58.960228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fde15-85ac-414d-b6bd-3fd4c760ee12 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:58.961565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:57:58.967584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba15c-3517-4d16-87a2-65b50134ed4b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:58.968022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba15c-3517-4d16-87a2-65b50134ed4b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:58.968251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba15c-3517-4d16-87a2-65b50134ed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:58.986738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-066ba15c-3517-4d16-87a2-65b50134ed4b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:57:58.987943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-066ba15c-3517-4d16-87a2-65b50134ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:57:58.995907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:57:59.035259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fde15-85ac-414d-b6bd-3fd4c760ee12 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:57:59.046985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fde15-85ac-414d-b6bd-3fd4c760ee12 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:57:59.050501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32cd76-1868-4058-8269-110f31950b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:57:59.051146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32cd76-1868-4058-8269-110f31950b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:57:59.051428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32cd76-1868-4058-8269-110f31950b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:57:59.051704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32cd76-1868-4058-8269-110f31950b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:57:59.061015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32cd76-1868-4058-8269-110f31950b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:57:59.062433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32cd76-1868-4058-8269-110f31950b2f service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:57:59.071697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 731/1467] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:57:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:58:01.346475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94049136-fa17-4dce-8b1f-48e628d1fea5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:01.347056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94049136-fa17-4dce-8b1f-48e628d1fea5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:01.347056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94049136-fa17-4dce-8b1f-48e628d1fea5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:01.348426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 11:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:01.353980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfafbc6-3652-4621-a4df-495299b26d53 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:01.354395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfafbc6-3652-4621-a4df-495299b26d53 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:01.354532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfafbc6-3652-4621-a4df-495299b26d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:01.372558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ccfafbc6-3652-4621-a4df-495299b26d53 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:01.373769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ccfafbc6-3652-4621-a4df-495299b26d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:01.381678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 11:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 11:58:01.419417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94049136-fa17-4dce-8b1f-48e628d1fea5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:58:01.430106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94049136-fa17-4dce-8b1f-48e628d1fea5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:58:01.433978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738579cb-a054-42dd-acb9-e9f7e01f8e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:58:01.434614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738579cb-a054-42dd-acb9-e9f7e01f8e29 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:01.434897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738579cb-a054-42dd-acb9-e9f7e01f8e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:01.435179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738579cb-a054-42dd-acb9-e9f7e01f8e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:01.444133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738579cb-a054-42dd-acb9-e9f7e01f8e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:58:01.445610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738579cb-a054-42dd-acb9-e9f7e01f8e29 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:58:01.455681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 738/1470] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:58:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:58:11.366722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=17 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:58:11.451609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=17 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:58:33.829716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfcb1-d07b-4a2b-bd0f-fcfd3b24fefa None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:33.830396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfcb1-d07b-4a2b-bd0f-fcfd3b24fefa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:33.830396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfcb1-d07b-4a2b-bd0f-fcfd3b24fefa None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:33.831681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 733/1471] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:33.837222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90cda-b93c-4a22-bfe0-2ed611f41a5b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:33.837679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90cda-b93c-4a22-bfe0-2ed611f41a5b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:33.837785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b90cda-b93c-4a22-bfe0-2ed611f41a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:33.855897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d5b90cda-b93c-4a22-bfe0-2ed611f41a5b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:33.857060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d5b90cda-b93c-4a22-bfe0-2ed611f41a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:33.864836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 739/1472] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:58:33.912567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65cd50-50e7-4d90-8952-862fabf54098 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:33.912861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65cd50-50e7-4d90-8952-862fabf54098 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:33.913143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65cd50-50e7-4d90-8952-862fabf54098 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:33.915217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 734/1473] 10.222.0.32 () {54 vars in 962 bytes} [Mon May 30 11:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:33.919918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bba143-8798-43b9-ae29-cb5fd89cd0c3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:33.920242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bba143-8798-43b9-ae29-cb5fd89cd0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:33.920525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bba143-8798-43b9-ae29-cb5fd89cd0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:33.940124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-74bba143-8798-43b9-ae29-cb5fd89cd0c3 None None] MFA Rules not processed for user `ad37bdce39f64836b0cd14613cabe91f`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:33.941344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-74bba143-8798-43b9-ae29-cb5fd89cd0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:33.947913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 740/1474] 10.222.0.32 () {58 vars in 1059 bytes} [Mon May 30 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:58:33.955955 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65cd50-50e7-4d90-8952-862fabf54098 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:58:33.971101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65cd50-50e7-4d90-8952-862fabf54098 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:58:33.974789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb657-bde5-49c0-b384-0976f0f18a58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad37bdce39f64836b0cd14613cabe91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:58:33.975451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb657-bde5-49c0-b384-0976f0f18a58 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:33.975736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb657-bde5-49c0-b384-0976f0f18a58 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:33.976017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb657-bde5-49c0-b384-0976f0f18a58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:33.987103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eb657-bde5-49c0-b384-0976f0f18a58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:58:33.988619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eb657-bde5-49c0-b384-0976f0f18a58 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:58:33.999346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 735/1475] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:58:36.385933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bdf24-650b-4fd5-8c57-37df2c40c4a7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:36.386477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bdf24-650b-4fd5-8c57-37df2c40c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:36.386600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bdf24-650b-4fd5-8c57-37df2c40c4a7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:36.387958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 741/1476] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:36.393765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8b57b-6b25-425c-a54b-5c587b41e6c4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:36.394204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8b57b-6b25-425c-a54b-5c587b41e6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:36.394351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8b57b-6b25-425c-a54b-5c587b41e6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:36.412064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3df8b57b-6b25-425c-a54b-5c587b41e6c4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:36.413304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3df8b57b-6b25-425c-a54b-5c587b41e6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:36.421413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 736/1477] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:58:36.446217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bdf24-650b-4fd5-8c57-37df2c40c4a7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:58:36.459204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bdf24-650b-4fd5-8c57-37df2c40c4a7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:58:36.463764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eb3a4-36d6-4358-b970-0f38ba4cd52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:58:36.464456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eb3a4-36d6-4358-b970-0f38ba4cd52c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:36.464749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eb3a4-36d6-4358-b970-0f38ba4cd52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:36.465032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eb3a4-36d6-4358-b970-0f38ba4cd52c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:36.465769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1eb3a4-36d6-4358-b970-0f38ba4cd52c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:58:36.467526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-6c1eb3a4-36d6-4358-b970-0f38ba4cd52c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 11:58:36.467526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 30 11:58:36.467526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: May 30 11:58:36.468714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1eb3a4-36d6-4358-b970-0f38ba4cd52c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:58:36.488913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 742/1478] 10.222.0.32 () {56 vars in 1108 bytes} [Mon May 30 11:58:36 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 11:58:38.798218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891992d-5f0f-46b9-bc4c-e85ddc259269 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:38.798790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891992d-5f0f-46b9-bc4c-e85ddc259269 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:38.798790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891992d-5f0f-46b9-bc4c-e85ddc259269 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:38.800142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 737/1479] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:38.805681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8421a8-b7fb-4c2f-9941-9930add6a8b0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:38.806151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8421a8-b7fb-4c2f-9941-9930add6a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:38.806249 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8421a8-b7fb-4c2f-9941-9930add6a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:38.825307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6c8421a8-b7fb-4c2f-9941-9930add6a8b0 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:38.826505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6c8421a8-b7fb-4c2f-9941-9930add6a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:38.834271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 743/1480] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:58:38.881612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891992d-5f0f-46b9-bc4c-e85ddc259269 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:58:38.896434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891992d-5f0f-46b9-bc4c-e85ddc259269 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:58:38.902540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac1a7d-0454-417b-a8bf-3880b7848a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:58:38.903181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1a7d-0454-417b-a8bf-3880b7848a93 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:38.903460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1a7d-0454-417b-a8bf-3880b7848a93 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:38.903736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac1a7d-0454-417b-a8bf-3880b7848a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:38.911866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac1a7d-0454-417b-a8bf-3880b7848a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:58:38.913475 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac1a7d-0454-417b-a8bf-3880b7848a93 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:58:38.923779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 738/1481] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 11:58:41.385389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae63e46-0fcf-4397-a6f4-1772fecf9d17 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:41.385953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae63e46-0fcf-4397-a6f4-1772fecf9d17 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:41.385953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae63e46-0fcf-4397-a6f4-1772fecf9d17 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:41.387332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 744/1482] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:41.392757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ce339-a6bd-4158-8486-c147c4e3ea53 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:41.393194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ce339-a6bd-4158-8486-c147c4e3ea53 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:41.393295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ce339-a6bd-4158-8486-c147c4e3ea53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:41.411860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ce6ce339-a6bd-4158-8486-c147c4e3ea53 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:41.413084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ce6ce339-a6bd-4158-8486-c147c4e3ea53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:41.421367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 739/1483] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:58:41.457505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae63e46-0fcf-4397-a6f4-1772fecf9d17 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:58:41.468864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae63e46-0fcf-4397-a6f4-1772fecf9d17 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:58:41.472488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b3a27-baa8-44e4-bb64-81a87df5071d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:58:41.473062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3a27-baa8-44e4-bb64-81a87df5071d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:41.473350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3a27-baa8-44e4-bb64-81a87df5071d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:41.473630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3a27-baa8-44e4-bb64-81a87df5071d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:41.482603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b3a27-baa8-44e4-bb64-81a87df5071d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:58:41.484042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b3a27-baa8-44e4-bb64-81a87df5071d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:58:41.493555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 745/1484] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:58:43.858771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38a70e-246e-46c3-9eb7-e2953d2f312c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:43.859301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38a70e-246e-46c3-9eb7-e2953d2f312c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:43.859301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38a70e-246e-46c3-9eb7-e2953d2f312c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:43.860660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 740/1485] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:43.865935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e597-4762-4b9d-bb8d-71dcd3034205 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:43.866351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e597-4762-4b9d-bb8d-71dcd3034205 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:43.866513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e597-4762-4b9d-bb8d-71dcd3034205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:43.885881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-55e0e597-4762-4b9d-bb8d-71dcd3034205 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:43.887107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55e0e597-4762-4b9d-bb8d-71dcd3034205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:43.895377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 746/1486] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:58:43.933518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38a70e-246e-46c3-9eb7-e2953d2f312c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:58:43.943996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38a70e-246e-46c3-9eb7-e2953d2f312c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:58:43.948368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00a221-4ecb-4d73-8f49-7a8c2a1af79d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:58:43.949000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a221-4ecb-4d73-8f49-7a8c2a1af79d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:43.949281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a221-4ecb-4d73-8f49-7a8c2a1af79d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:43.949554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a221-4ecb-4d73-8f49-7a8c2a1af79d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:43.958344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00a221-4ecb-4d73-8f49-7a8c2a1af79d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:58:43.959758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00a221-4ecb-4d73-8f49-7a8c2a1af79d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:58:43.969596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 741/1487] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:58:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:58:46.417572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfed6f-7e5f-455c-b4ff-c8d7f92ca25c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:46.418174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfed6f-7e5f-455c-b4ff-c8d7f92ca25c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:46.418174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfed6f-7e5f-455c-b4ff-c8d7f92ca25c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:46.419529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 747/1488] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:46.424917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe0a0-cd57-4b7b-a599-641a3be6e3b4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:46.425721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe0a0-cd57-4b7b-a599-641a3be6e3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:46.426243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe0a0-cd57-4b7b-a599-641a3be6e3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:46.445431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1c4fe0a0-cd57-4b7b-a599-641a3be6e3b4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:46.446999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1c4fe0a0-cd57-4b7b-a599-641a3be6e3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:46.455390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 742/1489] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:58:46.495565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfed6f-7e5f-455c-b4ff-c8d7f92ca25c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:58:46.509094 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfed6f-7e5f-455c-b4ff-c8d7f92ca25c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:58:46.513836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbabc47-e463-45cd-b8a5-59ecef56931e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:58:46.514488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabc47-e463-45cd-b8a5-59ecef56931e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:46.514768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabc47-e463-45cd-b8a5-59ecef56931e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:46.515045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbabc47-e463-45cd-b8a5-59ecef56931e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:46.525741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbabc47-e463-45cd-b8a5-59ecef56931e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:58:46.527231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbabc47-e463-45cd-b8a5-59ecef56931e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:58:46.537349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 748/1490] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:58:49.062297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071a93-fa1d-42f0-9390-5ca0e032996b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:49.062887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071a93-fa1d-42f0-9390-5ca0e032996b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:49.062887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071a93-fa1d-42f0-9390-5ca0e032996b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:49.064300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 743/1491] 10.222.0.32 () {54 vars in 926 bytes} [Mon May 30 11:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 11:58:49.069662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47be81-ac63-4e58-b3e2-546a1b112acc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:49.070143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47be81-ac63-4e58-b3e2-546a1b112acc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:49.070940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47be81-ac63-4e58-b3e2-546a1b112acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:49.088819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-da47be81-ac63-4e58-b3e2-546a1b112acc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 11:58:49.090007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-da47be81-ac63-4e58-b3e2-546a1b112acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 11:58:49.097710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 749/1492] 10.222.0.32 () {58 vars in 1023 bytes} [Mon May 30 11:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 11:58:49.137477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21071a93-fa1d-42f0-9390-5ca0e032996b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 11:58:49.148421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21071a93-fa1d-42f0-9390-5ca0e032996b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 11:58:49.151969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd8fd8-1894-474d-ba21-65b8add1e800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 11:58:49.152642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8fd8-1894-474d-ba21-65b8add1e800 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 11:58:49.152931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8fd8-1894-474d-ba21-65b8add1e800 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 11:58:49.153212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd8fd8-1894-474d-ba21-65b8add1e800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 11:58:49.162163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd8fd8-1894-474d-ba21-65b8add1e800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 11:58:49.163616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd8fd8-1894-474d-ba21-65b8add1e800 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 11:58:49.173270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 744/1493] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 11:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 11:58:59.083602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=33 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 11:58:59.168942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=33 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:00:17.006817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8d2ff-722c-4ca3-b346-9c2eada85a7d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.007388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8d2ff-722c-4ca3-b346-9c2eada85a7d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.007388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8d2ff-722c-4ca3-b346-9c2eada85a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.026231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-83b8d2ff-722c-4ca3-b346-9c2eada85a7d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:00:17.027417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-83b8d2ff-722c-4ca3-b346-9c2eada85a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:00:17.035117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 750/1494] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:00:17.043891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe49fd-b006-4af1-9aae-df17abb6e795 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.044427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe49fd-b006-4af1-9aae-df17abb6e795 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.044534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe49fd-b006-4af1-9aae-df17abb6e795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.062288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b1fe49fd-b006-4af1-9aae-df17abb6e795 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:00:17.063518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b1fe49fd-b006-4af1-9aae-df17abb6e795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:00:17.071552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 745/1495] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:00:17.076292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8d2ff-722c-4ca3-b346-9c2eada85a7d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.086754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8d2ff-722c-4ca3-b346-9c2eada85a7d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:17.091306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:17.092527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.092527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.092527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.093273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:00:17.093588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:17.094613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 30 12:00:17.094613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 30 12:00:17.094613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: May 30 12:00:17.095502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:17.107998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 751/1496] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:00:17.116276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe49fd-b006-4af1-9aae-df17abb6e795 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.125491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe49fd-b006-4af1-9aae-df17abb6e795 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:17.130010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:17.130990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.131277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.131557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.132292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:17.134038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:00:17.134038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) May 30 12:00:17.134038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: May 30 12:00:17.135240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:17.173777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 746/1497] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:00:17 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:00:17.178303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.188431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc9fb2-79e3-42b9-9c87-debf8ebfa93a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:17.244287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:17.244962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.245251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.245533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.246219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:17.247250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:00:17.247250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 30 12:00:17.247250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: May 30 12:00:17.248490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:17.268712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 752/1498] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:00:17 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:00:17.273917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.284237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7388aa6-6acf-44ff-b0a5-75572eb586e8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:17.288896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:17.289560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.289849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.290131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.290859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:17.291865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 30 12:00:17.291865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) May 30 12:00:17.291865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: May 30 12:00:17.292803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:17.305958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 747/1499] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:00:17.310070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.320876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47542d6c-b16c-41da-84d7-ba8ea32e8df2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:17.325443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e39654-1333-47ef-8b39-3dbcb5ca1dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:17.326104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e39654-1333-47ef-8b39-3dbcb5ca1dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.326391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e39654-1333-47ef-8b39-3dbcb5ca1dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.326671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e39654-1333-47ef-8b39-3dbcb5ca1dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.327400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e39654-1333-47ef-8b39-3dbcb5ca1dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:17.328448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-80e39654-1333-47ef-8b39-3dbcb5ca1dc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 30 12:00:17.328448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 30 12:00:17.328448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: May 30 12:00:17.329342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e39654-1333-47ef-8b39-3dbcb5ca1dc7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:17.343435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 753/1500] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:00:17.347537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.357991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef7951-78b7-4c20-a2ca-d949705e0105 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:17.363112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbc52b-55a5-4f6c-899e-3382c5104c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:17.363996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbc52b-55a5-4f6c-899e-3382c5104c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.364326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbc52b-55a5-4f6c-899e-3382c5104c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.364664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbc52b-55a5-4f6c-899e-3382c5104c06 admin admin] PATH_INFO: `/v3/projects/7af99df693a547278328fe2e36fdda41/users/ac744e14b4ee4c7b9b0042a8d4cdcd77/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.379591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbc52b-55a5-4f6c-899e-3382c5104c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af99df693a547278328fe2e36fdda41, user_id=ac744e14b4ee4c7b9b0042a8d4cdcd77, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:17.381333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-e2bbc52b-55a5-4f6c-899e-3382c5104c06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:00:17.381333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) May 30 12:00:17.381333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: May 30 12:00:17.383334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbc52b-55a5-4f6c-899e-3382c5104c06 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:17.396956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 748/1501] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:00:17 2022] PUT /identity/v3/projects/7af99df693a547278328fe2e36fdda41/users/ac744e14b4ee4c7b9b0042a8d4cdcd77/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:00:17.409103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b2bc9-175a-4c62-aec0-7e737e4beef3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.409398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b2bc9-175a-4c62-aec0-7e737e4beef3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.409685 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b2bc9-175a-4c62-aec0-7e737e4beef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.436380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a19b2bc9-175a-4c62-aec0-7e737e4beef3 None None] MFA Rules not processed for user `ac744e14b4ee4c7b9b0042a8d4cdcd77`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:00:17.437595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a19b2bc9-175a-4c62-aec0-7e737e4beef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:00:17.501773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 754/1502] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:00:17.515533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7da46-d11d-4d9b-a5d3-91c3444d3995 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.515827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7da46-d11d-4d9b-a5d3-91c3444d3995 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.516114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7da46-d11d-4d9b-a5d3-91c3444d3995 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.517539 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 749/1503] 10.222.0.32 () {54 vars in 963 bytes} [Mon May 30 12:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:00:17.523112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a73c1-eb49-4389-b0f7-afe257389a7e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.523371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a73c1-eb49-4389-b0f7-afe257389a7e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.523694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a73c1-eb49-4389-b0f7-afe257389a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.541366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b80a73c1-eb49-4389-b0f7-afe257389a7e None None] MFA Rules not processed for user `7c3c27e908e34977b248042f96c70dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:00:17.542568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b80a73c1-eb49-4389-b0f7-afe257389a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:00:17.586768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:00:17.595040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 755/1504] 10.222.0.32 () {58 vars in 1060 bytes} [Mon May 30 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:00:17.602745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7da46-d11d-4d9b-a5d3-91c3444d3995 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.615050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7da46-d11d-4d9b-a5d3-91c3444d3995 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:17.618792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3558624-5172-497c-89ae-883c743e3489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:17.619435 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3558624-5172-497c-89ae-883c743e3489 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.619718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3558624-5172-497c-89ae-883c743e3489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.619994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3558624-5172-497c-89ae-883c743e3489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.628374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3558624-5172-497c-89ae-883c743e3489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:17.629789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3558624-5172-497c-89ae-883c743e3489 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:17.639553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 750/1505] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:00:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:00:17.683964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43ced7-69b1-45f7-b276-11fa8d1e0308 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.684326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43ced7-69b1-45f7-b276-11fa8d1e0308 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.684580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43ced7-69b1-45f7-b276-11fa8d1e0308 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.685922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 756/1506] 10.222.0.32 () {52 vars in 829 bytes} [Mon May 30 12:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:00:17.689400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3558624-5172-497c-89ae-883c743e3489 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.751830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3558624-5172-497c-89ae-883c743e3489 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:17.757096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:17.757962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.758246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.758523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] PATH_INFO: `/v3/users/ac744e14b4ee4c7b9b0042a8d4cdcd77` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.759704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac744e14b4ee4c7b9b0042a8d4cdcd77)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:17.760741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:00:17.760741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) May 30 12:00:17.760741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: May 30 12:00:17.761944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:17.829140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac744e14b4ee4c7b9b0042a8d4cdcd77'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:17.847637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac744e14b4ee4c7b9b0042a8d4cdcd77'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:17.848469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac744e14b4ee4c7b9b0042a8d4cdcd77'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:17.861169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac744e14b4ee4c7b9b0042a8d4cdcd77'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:17.861847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac744e14b4ee4c7b9b0042a8d4cdcd77'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:17.866653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:00:17.881255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 751/1507] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:00:17 2022] DELETE /identity/v3/users/ac744e14b4ee4c7b9b0042a8d4cdcd77 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:00:17.885863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7e930-d17b-42cf-8712-5711d360b0bc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:17.886158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7e930-d17b-42cf-8712-5711d360b0bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:17.886444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7e930-d17b-42cf-8712-5711d360b0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:17.906378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-47c7e930-d17b-42cf-8712-5711d360b0bc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:00:17.907595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-47c7e930-d17b-42cf-8712-5711d360b0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:00:17.967213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 757/1508] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:00:17.978227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:17.979638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:00:18.041078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733808a-98d5-4274-ba1c-2bad4a3ac777 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:18.044708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b55568-d7cf-43d5-b1ff-8a819392434f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:18.045370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55568-d7cf-43d5-b1ff-8a819392434f service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:18.045655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55568-d7cf-43d5-b1ff-8a819392434f service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:18.045960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55568-d7cf-43d5-b1ff-8a819392434f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:18.055734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b55568-d7cf-43d5-b1ff-8a819392434f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:18.057255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b55568-d7cf-43d5-b1ff-8a819392434f service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:18.066950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 752/1509] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:00:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:00:18.418654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7e930-d17b-42cf-8712-5711d360b0bc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:00:18.420130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-47c7e930-d17b-42cf-8712-5711d360b0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:00:18.431995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7e930-d17b-42cf-8712-5711d360b0bc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:00:18.437043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:00:18.437947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:00:18.438470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:00:18.438578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] PATH_INFO: `/v3/projects/7af99df693a547278328fe2e36fdda41` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:00:18.441077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af99df693a547278328fe2e36fdda41)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:00:18.442058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:00:18.442058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 30 12:00:18.442058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: May 30 12:00:18.443395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:00:18.511003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af99df693a547278328fe2e36fdda41', 'request_id': 'req-b11e6856-ee00-4fd5-b662-a0baffc9d4be'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:18.520555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af99df693a547278328fe2e36fdda41', 'request_id': 'req-b11e6856-ee00-4fd5-b662-a0baffc9d4be'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:18.534524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af99df693a547278328fe2e36fdda41', 'request_id': 'req-b11e6856-ee00-4fd5-b662-a0baffc9d4be'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:18.546188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] The token cache is being invalidate because project 7af99df693a547278328fe2e36fdda41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:00:18.546481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:18.547223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b11e6856-ee00-4fd5-b662-a0baffc9d4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:00:18.548804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 758/1510] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:00:18 2022] DELETE /identity/v3/projects/7af99df693a547278328fe2e36fdda41 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:00:28.062676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:00:28.543149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:01:57.376447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b829-7660-4e3f-b4c7-b4a84de536f8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:01:57.377069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b829-7660-4e3f-b4c7-b4a84de536f8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:01:57.377069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b829-7660-4e3f-b4c7-b4a84de536f8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:01:57.378367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 12:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:01:57.383373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842638a-beae-436b-b010-18d13628b7ad None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:01:57.383816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842638a-beae-436b-b010-18d13628b7ad None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:01:57.383924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842638a-beae-436b-b010-18d13628b7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:01:57.402514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d842638a-beae-436b-b010-18d13628b7ad None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:01:57.403723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d842638a-beae-436b-b010-18d13628b7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:01:57.461856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 12:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:01:57.641378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80b829-7660-4e3f-b4c7-b4a84de536f8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:01:57.643134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ad80b829-7660-4e3f-b4c7-b4a84de536f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:01:57.718332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80b829-7660-4e3f-b4c7-b4a84de536f8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:01:57.724061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ebd66-eb40-43a6-b440-e02d16f59995 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:01:57.724798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ebd66-eb40-43a6-b440-e02d16f59995 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:01:57.725086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ebd66-eb40-43a6-b440-e02d16f59995 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:01:57.725379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ebd66-eb40-43a6-b440-e02d16f59995 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:01:57.734400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ebd66-eb40-43a6-b440-e02d16f59995 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:01:57.735989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ebd66-eb40-43a6-b440-e02d16f59995 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:01:57.745611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 754/1513] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:01:58.644027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99114-3a53-4458-9b3f-4e61a6c3cf6f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:01:58.644712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99114-3a53-4458-9b3f-4e61a6c3cf6f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:01:58.644712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c99114-3a53-4458-9b3f-4e61a6c3cf6f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:01:58.646263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 12:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:01:58.651095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90a85-81b8-4369-a852-6d3b5de53eb7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:01:58.651531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90a85-81b8-4369-a852-6d3b5de53eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:01:58.651657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca90a85-81b8-4369-a852-6d3b5de53eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:01:58.671167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-aca90a85-81b8-4369-a852-6d3b5de53eb7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:01:58.672427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-aca90a85-81b8-4369-a852-6d3b5de53eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:01:58.680911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 12:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:01:58.797626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69ed8-82a2-456b-879c-81b9aff3e66e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:01:58.797947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69ed8-82a2-456b-879c-81b9aff3e66e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:01:58.798238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69ed8-82a2-456b-879c-81b9aff3e66e None None] PATH_INFO: `/v3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:01:58.799584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 761/1516] 10.222.0.65 () {54 vars in 925 bytes} [Mon May 30 12:01:58 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:01:58.812962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20def-e9b1-430c-afec-2ea41c2e65e0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:01:58.813305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20def-e9b1-430c-afec-2ea41c2e65e0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:01:58.813622 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20def-e9b1-430c-afec-2ea41c2e65e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:01:58.832004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b4f20def-e9b1-430c-afec-2ea41c2e65e0 None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:01:58.833253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b4f20def-e9b1-430c-afec-2ea41c2e65e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:01:58.839755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e69ed8-82a2-456b-879c-81b9aff3e66e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:01:58.855512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e69ed8-82a2-456b-879c-81b9aff3e66e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:01:58.863023 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e438ff7-10a4-4bb1-8d72-66626a040ee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:01:58.863679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438ff7-10a4-4bb1-8d72-66626a040ee0 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:01:58.863997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438ff7-10a4-4bb1-8d72-66626a040ee0 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:01:58.864320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e438ff7-10a4-4bb1-8d72-66626a040ee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:01:58.875699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e438ff7-10a4-4bb1-8d72-66626a040ee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:01:58.877380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e438ff7-10a4-4bb1-8d72-66626a040ee0 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:01:58.888824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 762/1517] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:01:58.904525 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 756/1518] 10.222.0.65 () {58 vars in 1015 bytes} [Mon May 30 12:01:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:02:02.941822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e56cf-1d85-45f3-9443-1386bdd2e269 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:02.942377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e56cf-1d85-45f3-9443-1386bdd2e269 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:02.942377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e56cf-1d85-45f3-9443-1386bdd2e269 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:02.943750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 763/1519] 10.222.0.32 () {54 vars in 960 bytes} [Mon May 30 12:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 12:02:02.948861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ea550-b14a-4f46-b294-67c630dd82d6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:02.949319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ea550-b14a-4f46-b294-67c630dd82d6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:02.949432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ea550-b14a-4f46-b294-67c630dd82d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:02.968569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c60ea550-b14a-4f46-b294-67c630dd82d6 None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:02:02.969743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c60ea550-b14a-4f46-b294-67c630dd82d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:02.978988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 757/1520] 10.222.0.32 () {58 vars in 1057 bytes} [Mon May 30 12:02:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:02:02.986383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e56cf-1d85-45f3-9443-1386bdd2e269 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:02:03.000175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e56cf-1d85-45f3-9443-1386bdd2e269 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:02:03.005272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6f720-d821-4f65-a4dc-c63ef60ccdd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:02:03.005921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6f720-d821-4f65-a4dc-c63ef60ccdd2 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:03.006213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6f720-d821-4f65-a4dc-c63ef60ccdd2 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:03.006625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6f720-d821-4f65-a4dc-c63ef60ccdd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:03.015538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6f720-d821-4f65-a4dc-c63ef60ccdd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:02:03.017165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6f720-d821-4f65-a4dc-c63ef60ccdd2 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:02:03.028138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 764/1521] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:02:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:02:10.203734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8755598-9b76-405a-a1ec-72aa66071ffb None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:10.204345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8755598-9b76-405a-a1ec-72aa66071ffb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:10.204345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8755598-9b76-405a-a1ec-72aa66071ffb None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:10.205745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 12:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:02:10.211149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce14164-81fc-43e3-9f8e-8a6c88e649fa None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:10.211589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce14164-81fc-43e3-9f8e-8a6c88e649fa None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:10.211779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce14164-81fc-43e3-9f8e-8a6c88e649fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:10.230750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2ce14164-81fc-43e3-9f8e-8a6c88e649fa None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:02:10.231964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2ce14164-81fc-43e3-9f8e-8a6c88e649fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:10.239928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 12:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:02:10.397835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8755598-9b76-405a-a1ec-72aa66071ffb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:02:10.411980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8755598-9b76-405a-a1ec-72aa66071ffb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:02:10.418408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2db487-de86-4cb9-bbe7-512a2ab90ddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:02:10.419045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db487-de86-4cb9-bbe7-512a2ab90ddb service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:10.419327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db487-de86-4cb9-bbe7-512a2ab90ddb service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:10.419604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db487-de86-4cb9-bbe7-512a2ab90ddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:10.427702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2db487-de86-4cb9-bbe7-512a2ab90ddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:02:10.429646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2db487-de86-4cb9-bbe7-512a2ab90ddb service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:02:10.438918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 759/1524] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:02:11.317074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698979-85c6-4974-a2cc-94fe15ac1084 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:11.317619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698979-85c6-4974-a2cc-94fe15ac1084 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:11.317740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698979-85c6-4974-a2cc-94fe15ac1084 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:11.320033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 30 12:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:02:11.325361 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aca6-59ed-4849-9891-87b8ab5da9dc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:11.325929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aca6-59ed-4849-9891-87b8ab5da9dc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:11.325929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aca6-59ed-4849-9891-87b8ab5da9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:11.347327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a357aca6-59ed-4849-9891-87b8ab5da9dc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:02:11.348775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a357aca6-59ed-4849-9891-87b8ab5da9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:11.356900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 30 12:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:02:11.508652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6698979-85c6-4974-a2cc-94fe15ac1084 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:02:11.523998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6698979-85c6-4974-a2cc-94fe15ac1084 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:02:11.529545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4cb46-64f4-4bec-8ef0-d6a1b4d7cf0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:02:11.530187 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4cb46-64f4-4bec-8ef0-d6a1b4d7cf0b service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:11.530479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4cb46-64f4-4bec-8ef0-d6a1b4d7cf0b service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:11.530765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4cb46-64f4-4bec-8ef0-d6a1b4d7cf0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:11.539612 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4cb46-64f4-4bec-8ef0-d6a1b4d7cf0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:02:11.541238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4cb46-64f4-4bec-8ef0-d6a1b4d7cf0b service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:02:11.550612 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 767/1527] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:02:15.630889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e31b46-4fdc-484f-8c2d-ab10cfcc837c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:15.631926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e31b46-4fdc-484f-8c2d-ab10cfcc837c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:15.632374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e31b46-4fdc-484f-8c2d-ab10cfcc837c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:15.633786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 12:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:02:15.639232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aeebd-619d-4830-a42d-ab5e8df600b6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:15.639651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aeebd-619d-4830-a42d-ab5e8df600b6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:15.639794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aeebd-619d-4830-a42d-ab5e8df600b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:15.658769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7a6aeebd-619d-4830-a42d-ab5e8df600b6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:02:15.660006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7a6aeebd-619d-4830-a42d-ab5e8df600b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:15.668150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 12:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:02:15.705839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e31b46-4fdc-484f-8c2d-ab10cfcc837c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:02:15.707514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f1e31b46-4fdc-484f-8c2d-ab10cfcc837c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:15.763364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:02:15.767883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e31b46-4fdc-484f-8c2d-ab10cfcc837c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:02:15.771699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5870855-ff0f-49f0-8847-28c2363eaf40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:02:15.772566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870855-ff0f-49f0-8847-28c2363eaf40 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:15.772852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870855-ff0f-49f0-8847-28c2363eaf40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:15.773131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870855-ff0f-49f0-8847-28c2363eaf40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:15.782073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5870855-ff0f-49f0-8847-28c2363eaf40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:02:15.783504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5870855-ff0f-49f0-8847-28c2363eaf40 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:02:15.793181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 762/1530] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:02:18.067193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b4891-42cd-4003-be0c-70fae03cd793 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:18.067759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b4891-42cd-4003-be0c-70fae03cd793 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:18.067872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b4891-42cd-4003-be0c-70fae03cd793 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:18.070047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 12:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:02:18.075396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013ce49-d3bf-4a82-8bdc-ef3c66033177 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:18.075845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013ce49-d3bf-4a82-8bdc-ef3c66033177 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:18.075961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013ce49-d3bf-4a82-8bdc-ef3c66033177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:18.093778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5013ce49-d3bf-4a82-8bdc-ef3c66033177 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:02:18.094988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5013ce49-d3bf-4a82-8bdc-ef3c66033177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:18.103515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 12:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:02:18.139957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b4891-42cd-4003-be0c-70fae03cd793 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:02:18.151502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b4891-42cd-4003-be0c-70fae03cd793 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:02:18.155125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3afd2-ca5d-4632-8224-3a3288ff6397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:02:18.155779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3afd2-ca5d-4632-8224-3a3288ff6397 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:18.156071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3afd2-ca5d-4632-8224-3a3288ff6397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:18.156396 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3afd2-ca5d-4632-8224-3a3288ff6397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:18.165640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3afd2-ca5d-4632-8224-3a3288ff6397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:02:18.167094 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3afd2-ca5d-4632-8224-3a3288ff6397 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:02:18.176463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 770/1533] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:02:22.010251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f6965-086f-4059-bfde-c6260b3a71ea None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:22.010763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f6965-086f-4059-bfde-c6260b3a71ea None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:22.010763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f6965-086f-4059-bfde-c6260b3a71ea None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:22.012100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 12:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:02:22.017327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be510-93d2-4dc4-97ef-de040390a32e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:22.017711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be510-93d2-4dc4-97ef-de040390a32e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:22.017917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be510-93d2-4dc4-97ef-de040390a32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:22.025254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=58 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:02:22.038569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-562be510-93d2-4dc4-97ef-de040390a32e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:02:22.039781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-562be510-93d2-4dc4-97ef-de040390a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:22.047718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:02:22.322207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f6965-086f-4059-bfde-c6260b3a71ea None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:02:22.323900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a68f6965-086f-4059-bfde-c6260b3a71ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:22.381858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f6965-086f-4059-bfde-c6260b3a71ea None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:02:22.385411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b9cb4-6572-44b9-bbd6-4b5c42632f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:02:22.386058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b9cb4-6572-44b9-bbd6-4b5c42632f3b service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:22.386345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b9cb4-6572-44b9-bbd6-4b5c42632f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:22.386624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b9cb4-6572-44b9-bbd6-4b5c42632f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:22.395987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b9cb4-6572-44b9-bbd6-4b5c42632f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:02:22.398018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b9cb4-6572-44b9-bbd6-4b5c42632f3b service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:02:22.407666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 765/1536] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:02:24.725818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e63855-7c45-4449-a1f4-5903f04ec916 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:24.726374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e63855-7c45-4449-a1f4-5903f04ec916 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:24.726374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e63855-7c45-4449-a1f4-5903f04ec916 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:24.728038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 30 12:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:02:24.733136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f50c5-cf1b-4b0d-86a8-098363aca07e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:24.733597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f50c5-cf1b-4b0d-86a8-098363aca07e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:24.733764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f50c5-cf1b-4b0d-86a8-098363aca07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:24.752955 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-789f50c5-cf1b-4b0d-86a8-098363aca07e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:02:24.754149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-789f50c5-cf1b-4b0d-86a8-098363aca07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:24.762749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 30 12:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:02:25.041840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e63855-7c45-4449-a1f4-5903f04ec916 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:02:25.043538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a1e63855-7c45-4449-a1f4-5903f04ec916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:02:25.057413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e63855-7c45-4449-a1f4-5903f04ec916 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:02:25.060976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdc66f-f183-4f58-82a8-bd4dc8b26905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:02:25.061628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc66f-f183-4f58-82a8-bd4dc8b26905 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:02:25.061932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc66f-f183-4f58-82a8-bd4dc8b26905 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:02:25.062245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc66f-f183-4f58-82a8-bd4dc8b26905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:02:25.071176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdc66f-f183-4f58-82a8-bd4dc8b26905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:02:25.072650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdc66f-f183-4f58-82a8-bd4dc8b26905 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:02:25.082092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 773/1539] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:02:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:02:34.748000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=36 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:02:35.077850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=8 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:03:57.959010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e60ec-7dcb-42f6-9e40-415252f59746 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:57.959612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e60ec-7dcb-42f6-9e40-415252f59746 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:57.959612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e60ec-7dcb-42f6-9e40-415252f59746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:57.990541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4b8e60ec-7dcb-42f6-9e40-415252f59746 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:57.992212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4b8e60ec-7dcb-42f6-9e40-415252f59746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:58.005740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 767/1540] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:58.015051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a43ac6-61d8-4ba0-8b18-98c09681df63 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.015629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a43ac6-61d8-4ba0-8b18-98c09681df63 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.015629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a43ac6-61d8-4ba0-8b18-98c09681df63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.045569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-51a43ac6-61d8-4ba0-8b18-98c09681df63 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:58.046799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-51a43ac6-61d8-4ba0-8b18-98c09681df63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:58.059237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 774/1541] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:58.064432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e60ec-7dcb-42f6-9e40-415252f59746 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.077679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e60ec-7dcb-42f6-9e40-415252f59746 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.086760 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.087764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.088434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.088942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.089890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:03:58.090416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.091643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 30 12:03:58.091643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) May 30 12:03:58.091643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: May 30 12:03:58.092991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.110757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 768/1542] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:03:58.119825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a43ac6-61d8-4ba0-8b18-98c09681df63 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.131517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a43ac6-61d8-4ba0-8b18-98c09681df63 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.148028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.149279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.149716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.150135 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.151349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.159515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:03:58.159515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 30 12:03:58.159515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: May 30 12:03:58.161025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.208831 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 775/1543] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:03:58.213269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.230731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6902da-f139-4f9d-94e5-26e73fe3bb2d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.318045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.319169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.319683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.320206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.332920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.334493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:03:58.334493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) May 30 12:03:58.334493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: May 30 12:03:58.336303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.365779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 769/1544] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:03:58.370196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.392193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87c283-abcd-4baa-b3c3-41f951147c66 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.405060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1968c-8c6e-493e-98ec-a59f2c8ba9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.406037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1968c-8c6e-493e-98ec-a59f2c8ba9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.406496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1968c-8c6e-493e-98ec-a59f2c8ba9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.406948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1968c-8c6e-493e-98ec-a59f2c8ba9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.407861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1968c-8c6e-493e-98ec-a59f2c8ba9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.409752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1968c-8c6e-493e-98ec-a59f2c8ba9c9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.429838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 776/1545] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:03:58.434083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.452140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da70a8b-7639-469b-a0c5-6c4c99c51508 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.460875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb73f43-adc4-4920-9361-ee0f0e208160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.461722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb73f43-adc4-4920-9361-ee0f0e208160 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.462111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb73f43-adc4-4920-9361-ee0f0e208160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.462492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb73f43-adc4-4920-9361-ee0f0e208160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.463399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb73f43-adc4-4920-9361-ee0f0e208160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.465334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb73f43-adc4-4920-9361-ee0f0e208160 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.481839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 770/1546] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:03:58.482491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ac7f1-a991-4a65-91e1-2a3fdefd4e4b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.483019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ac7f1-a991-4a65-91e1-2a3fdefd4e4b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.483546 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ac7f1-a991-4a65-91e1-2a3fdefd4e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.488181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb73f43-adc4-4920-9361-ee0f0e208160 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.496992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb73f43-adc4-4920-9361-ee0f0e208160 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.512978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199cfde-aaaf-4f9a-9e4a-f3aa9861d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.513901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cfde-aaaf-4f9a-9e4a-f3aa9861d4be admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.514192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cfde-aaaf-4f9a-9e4a-f3aa9861d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.514476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199cfde-aaaf-4f9a-9e4a-f3aa9861d4be admin admin] PATH_INFO: `/v3/projects/8b5074cf3e6248668995f266845c8969/users/c76e8d7bfdc04ea79c83ca7fb6b9ade1/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.515943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d14ac7f1-a991-4a65-91e1-2a3fdefd4e4b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:58.517196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d14ac7f1-a991-4a65-91e1-2a3fdefd4e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:58.538052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 777/1547] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:03:58.546081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6f0f-423b-4fc1-a8c5-88a28a9c156f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.546390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6f0f-423b-4fc1-a8c5-88a28a9c156f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.546770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6f0f-423b-4fc1-a8c5-88a28a9c156f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.557086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199cfde-aaaf-4f9a-9e4a-f3aa9861d4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5074cf3e6248668995f266845c8969, user_id=c76e8d7bfdc04ea79c83ca7fb6b9ade1, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.560414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199cfde-aaaf-4f9a-9e4a-f3aa9861d4be admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.583664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bcaf6f0f-423b-4fc1-a8c5-88a28a9c156f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:58.584926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bcaf6f0f-423b-4fc1-a8c5-88a28a9c156f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:58.598024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 771/1548] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:03:58 2022] PUT /identity/v3/projects/8b5074cf3e6248668995f266845c8969/users/c76e8d7bfdc04ea79c83ca7fb6b9ade1/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:03:58.603693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc514c-23eb-47b6-929f-a218dc59e189 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.604030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc514c-23eb-47b6-929f-a218dc59e189 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.604353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc514c-23eb-47b6-929f-a218dc59e189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.643007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6dfc514c-23eb-47b6-929f-a218dc59e189 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:58.645752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6dfc514c-23eb-47b6-929f-a218dc59e189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:58.757541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 778/1549] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:03:58.762081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf6f0f-423b-4fc1-a8c5-88a28a9c156f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.768004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 772/1550] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:58.782516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf6f0f-423b-4fc1-a8c5-88a28a9c156f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.784579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc514c-23eb-47b6-929f-a218dc59e189 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.817768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.818459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.818744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.819018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.819760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:03:58.820071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.821731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.852330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 779/1551] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 12:03:58.861046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:58.872950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:03:58.883416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccca266-ef67-4490-9165-e08cd32e97a6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.897468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9306434-9d20-4934-9c0b-d9179637071d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.898153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9306434-9d20-4934-9c0b-d9179637071d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.898444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9306434-9d20-4934-9c0b-d9179637071d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.898722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9306434-9d20-4934-9c0b-d9179637071d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.899428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9306434-9d20-4934-9c0b-d9179637071d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.901393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9306434-9d20-4934-9c0b-d9179637071d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.910093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc514c-23eb-47b6-929f-a218dc59e189 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:58.915905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a019c-84af-444e-b01b-69bb3b67c782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:58.916613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a019c-84af-444e-b01b-69bb3b67c782 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.916902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a019c-84af-444e-b01b-69bb3b67c782 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.917187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a019c-84af-444e-b01b-69bb3b67c782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.933729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a019c-84af-444e-b01b-69bb3b67c782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:58.935256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a019c-84af-444e-b01b-69bb3b67c782 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:58.958143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 773/1552] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:03:58.959306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2166af9-b50f-4961-abec-67bb282726e9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.959884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2166af9-b50f-4961-abec-67bb282726e9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.959884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2166af9-b50f-4961-abec-67bb282726e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:58.994977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 780/1553] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:03:58.996630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c25eb8-c252-47d7-8baf-47576cc4f0fb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:58.996977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c25eb8-c252-47d7-8baf-47576cc4f0fb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:58.997263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c25eb8-c252-47d7-8baf-47576cc4f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.017576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e2166af9-b50f-4961-abec-67bb282726e9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.019371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e2166af9-b50f-4961-abec-67bb282726e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.069441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-93c25eb8-c252-47d7-8baf-47576cc4f0fb None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.070687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93c25eb8-c252-47d7-8baf-47576cc4f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.201173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 774/1554] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.202554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2166af9-b50f-4961-abec-67bb282726e9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.217254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2166af9-b50f-4961-abec-67bb282726e9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.218273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 781/1555] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.219797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653823fb-8bf1-4970-b65d-e4e8b5aabf08 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.220040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653823fb-8bf1-4970-b65d-e4e8b5aabf08 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.220360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653823fb-8bf1-4970-b65d-e4e8b5aabf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.223961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5437c-b628-4ef1-94a2-93b23e5a267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.224980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5437c-b628-4ef1-94a2-93b23e5a267f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.226125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5437c-b628-4ef1-94a2-93b23e5a267f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.226623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5437c-b628-4ef1-94a2-93b23e5a267f admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.227541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5437c-b628-4ef1-94a2-93b23e5a267f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.229738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5437c-b628-4ef1-94a2-93b23e5a267f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.255022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-653823fb-8bf1-4970-b65d-e4e8b5aabf08 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.256448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-653823fb-8bf1-4970-b65d-e4e8b5aabf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.260285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 775/1556] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/users => generated 291 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:03:59.262151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0d17a-baaf-44e7-ac67-b7af283fdb31 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.263246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0d17a-baaf-44e7-ac67-b7af283fdb31 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.264969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0d17a-baaf-44e7-ac67-b7af283fdb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.283268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 782/1557] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.283268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862728-d77d-46f5-b4a2-271561b23674 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.283268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862728-d77d-46f5-b4a2-271561b23674 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.283268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862728-d77d-46f5-b4a2-271561b23674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.312045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-72e0d17a-baaf-44e7-ac67-b7af283fdb31 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.313315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-72e0d17a-baaf-44e7-ac67-b7af283fdb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.314533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8c862728-d77d-46f5-b4a2-271561b23674 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.316258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8c862728-d77d-46f5-b4a2-271561b23674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.325049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 783/1558] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.326502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dd599-c118-4b13-8366-10a9d9368b36 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.326795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dd599-c118-4b13-8366-10a9d9368b36 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.327084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dd599-c118-4b13-8366-10a9d9368b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.332885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 776/1559] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.334225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0d17a-baaf-44e7-ac67-b7af283fdb31 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.345841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:03:59.351900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0d17a-baaf-44e7-ac67-b7af283fdb31 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.354740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2b7dd599-c118-4b13-8366-10a9d9368b36 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.359161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b7dd599-c118-4b13-8366-10a9d9368b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.362125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f9e31-5193-4208-b04c-9accb708286a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.363050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f9e31-5193-4208-b04c-9accb708286a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.363346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f9e31-5193-4208-b04c-9accb708286a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.363632 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f9e31-5193-4208-b04c-9accb708286a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.364399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f9e31-5193-4208-b04c-9accb708286a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.366177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f9e31-5193-4208-b04c-9accb708286a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.367648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 784/1560] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.369211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc26b7-acef-4558-ba79-0bff9751d71e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.369619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc26b7-acef-4558-ba79-0bff9751d71e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.369834 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc26b7-acef-4558-ba79-0bff9751d71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.381910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 777/1561] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:03:59.383240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7924a-6a3d-4337-8bb7-7d337e2e3126 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.383538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7924a-6a3d-4337-8bb7-7d337e2e3126 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.383821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7924a-6a3d-4337-8bb7-7d337e2e3126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.395747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4bfc26b7-acef-4558-ba79-0bff9751d71e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.397356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4bfc26b7-acef-4558-ba79-0bff9751d71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.404439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0ee7924a-6a3d-4337-8bb7-7d337e2e3126 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.405639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ee7924a-6a3d-4337-8bb7-7d337e2e3126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.406872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 785/1562] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.407730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc26b7-acef-4558-ba79-0bff9751d71e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.417549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc26b7-acef-4558-ba79-0bff9751d71e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.422738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66547c5-84b4-49fe-87c1-a3df9a4a01a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.422881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 778/1563] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.426617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a35f-0ac8-48ce-8a7a-b972ff20a950 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.426617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a35f-0ac8-48ce-8a7a-b972ff20a950 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.426617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325a35f-0ac8-48ce-8a7a-b972ff20a950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.428292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66547c5-84b4-49fe-87c1-a3df9a4a01a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.428794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66547c5-84b4-49fe-87c1-a3df9a4a01a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.429274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66547c5-84b4-49fe-87c1-a3df9a4a01a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.430220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66547c5-84b4-49fe-87c1-a3df9a4a01a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:03:59.430717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66547c5-84b4-49fe-87c1-a3df9a4a01a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.433740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66547c5-84b4-49fe-87c1-a3df9a4a01a0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.445220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b325a35f-0ac8-48ce-8a7a-b972ff20a950 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.446770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b325a35f-0ac8-48ce-8a7a-b972ff20a950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.455233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 786/1564] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:03:59.455233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d8c7-7376-4a74-923c-086a64e5d963 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.455233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d8c7-7376-4a74-923c-086a64e5d963 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.455233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d8c7-7376-4a74-923c-086a64e5d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.469634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 779/1565] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.469634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a35f-0ac8-48ce-8a7a-b972ff20a950 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.479907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325a35f-0ac8-48ce-8a7a-b972ff20a950 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.484858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.485900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.487029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.487029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.487299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:03:59.487773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.489551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.503972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 780/1566] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:03:59.505013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.505934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8996d8c7-7376-4a74-923c-086a64e5d963 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.507286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8996d8c7-7376-4a74-923c-086a64e5d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.520971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 787/1567] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.523342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377c3a-9df0-457e-be3b-a828fd423a6e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.523761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377c3a-9df0-457e-be3b-a828fd423a6e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.524121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377c3a-9df0-457e-be3b-a828fd423a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.538485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a5dc4-87a3-4301-b032-e64883c292ef admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.545451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7032c-21e1-4384-a967-116de0a06bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.546220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7032c-21e1-4384-a967-116de0a06bec admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.546580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7032c-21e1-4384-a967-116de0a06bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.546928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7032c-21e1-4384-a967-116de0a06bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.547740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7032c-21e1-4384-a967-116de0a06bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.548741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7d377c3a-9df0-457e-be3b-a828fd423a6e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.550728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7032c-21e1-4384-a967-116de0a06bec admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.550875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7d377c3a-9df0-457e-be3b-a828fd423a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.565179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 788/1568] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.565179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d377c3a-9df0-457e-be3b-a828fd423a6e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.567577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 781/1569] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:03:59.568487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7032c-21e1-4384-a967-116de0a06bec admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.578564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d377c3a-9df0-457e-be3b-a828fd423a6e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.581403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7032c-21e1-4384-a967-116de0a06bec admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.584292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.585349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.586054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.586597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7414d-7bfe-4443-a085-448e10103333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.588310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.588436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7414d-7bfe-4443-a085-448e10103333 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.588436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7414d-7bfe-4443-a085-448e10103333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.588436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7414d-7bfe-4443-a085-448e10103333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.588680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:03:59.588759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7414d-7bfe-4443-a085-448e10103333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.588844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.590460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7414d-7bfe-4443-a085-448e10103333 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.591252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.608563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 789/1570] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:03:59.610400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.624397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47677618-acb1-401a-9229-ac08e2e6e439 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.625015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 782/1571] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:03:59.625785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7414d-7bfe-4443-a085-448e10103333 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.680521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:03:59.704058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae84389-d1d6-4dd2-8822-d41f6aa98279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.705154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84389-d1d6-4dd2-8822-d41f6aa98279 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.705438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84389-d1d6-4dd2-8822-d41f6aa98279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.705712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84389-d1d6-4dd2-8822-d41f6aa98279 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.707081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae84389-d1d6-4dd2-8822-d41f6aa98279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:03:59.707437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae84389-d1d6-4dd2-8822-d41f6aa98279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.707586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7414d-7bfe-4443-a085-448e10103333 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.709184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae84389-d1d6-4dd2-8822-d41f6aa98279 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.712902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635e06-7c79-4852-beb4-a825a94d7b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.713584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635e06-7c79-4852-beb4-a825a94d7b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.713873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635e06-7c79-4852-beb4-a825a94d7b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.714156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635e06-7c79-4852-beb4-a825a94d7b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.714858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635e06-7c79-4852-beb4-a825a94d7b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.717419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635e06-7c79-4852-beb4-a825a94d7b23 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.721526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 790/1572] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:03:59.724068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe295523-c590-475a-a93c-482a62df83ca None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.724068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe295523-c590-475a-a93c-482a62df83ca None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.724318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe295523-c590-475a-a93c-482a62df83ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.746596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fe295523-c590-475a-a93c-482a62df83ca None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:03:59.747824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fe295523-c590-475a-a93c-482a62df83ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:03:59.756283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 791/1573] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:03:59.757079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe295523-c590-475a-a93c-482a62df83ca None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.765613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 783/1574] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/projects => generated 393 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:03:59.766688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635e06-7c79-4852-beb4-a825a94d7b23 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.767119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe295523-c590-475a-a93c-482a62df83ca None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.841562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.842162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.842962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.843261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.844060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:03:59.844580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.846220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.847903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635e06-7c79-4852-beb4-a825a94d7b23 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.855400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09a118-34dd-4e42-814c-1407c764fdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.856382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a118-34dd-4e42-814c-1407c764fdee admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.856677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a118-34dd-4e42-814c-1407c764fdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.857421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09a118-34dd-4e42-814c-1407c764fdee admin admin] PATH_INFO: `/v3/projects/96630b03d2094e1494969734ef6cd2a6/users/829c8ebd4eaa4b47b130ce6c2a8f80e5/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.860601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 792/1575] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:03:59.861238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.880288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59cb0-bf81-4c5c-97d3-88712d721af3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:03:59.882187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09a118-34dd-4e42-814c-1407c764fdee admin admin] RBAC: Authorizing `identity:create_grant(project_id=96630b03d2094e1494969734ef6cd2a6, user_id=829c8ebd4eaa4b47b130ce6c2a8f80e5, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.885643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09a118-34dd-4e42-814c-1407c764fdee admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.885923 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd1b99-b964-4c9d-83cc-63054f8325c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:03:59.886677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1b99-b964-4c9d-83cc-63054f8325c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:03:59.887198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1b99-b964-4c9d-83cc-63054f8325c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:03:59.887547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd1b99-b964-4c9d-83cc-63054f8325c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:03:59.888382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd1b99-b964-4c9d-83cc-63054f8325c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:03:59.890877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd1b99-b964-4c9d-83cc-63054f8325c3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:03:59.907278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 784/1576] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:03:59 2022] PUT /identity/v3/projects/96630b03d2094e1494969734ef6cd2a6/users/829c8ebd4eaa4b47b130ce6c2a8f80e5/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:03:59.908818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09a118-34dd-4e42-814c-1407c764fdee admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.941903 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 793/1577] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:03:59.943143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd1b99-b964-4c9d-83cc-63054f8325c3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:03:59.965120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:00.025663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09a118-34dd-4e42-814c-1407c764fdee admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.031672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583585-9dee-4e41-a918-b4b5cd9c145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.032608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583585-9dee-4e41-a918-b4b5cd9c145c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.032901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583585-9dee-4e41-a918-b4b5cd9c145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.033339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583585-9dee-4e41-a918-b4b5cd9c145c admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.034359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583585-9dee-4e41-a918-b4b5cd9c145c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.037421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583585-9dee-4e41-a918-b4b5cd9c145c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.041485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd1b99-b964-4c9d-83cc-63054f8325c3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.058433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d04a2f-284b-46cd-88a2-f8e1ddb143e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.059712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d04a2f-284b-46cd-88a2-f8e1ddb143e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.060208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d04a2f-284b-46cd-88a2-f8e1ddb143e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.060872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d04a2f-284b-46cd-88a2-f8e1ddb143e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.061782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d04a2f-284b-46cd-88a2-f8e1ddb143e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.064511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d04a2f-284b-46cd-88a2-f8e1ddb143e0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.070717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 785/1578] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.072417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583585-9dee-4e41-a918-b4b5cd9c145c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.088270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583585-9dee-4e41-a918-b4b5cd9c145c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.093500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.094182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.094466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.094746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.095624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:00.095940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.098636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.107185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 794/1579] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:03:59 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.109167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d04a2f-284b-46cd-88a2-f8e1ddb143e0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.120337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 786/1580] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:00.120337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.126729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227d983-aa64-4eba-a140-3c723a9e4c1e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.166537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:00.212643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d04a2f-284b-46cd-88a2-f8e1ddb143e0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.219353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b3037-4655-469a-8464-a654d4e87c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.219612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daefee1-146b-442d-80d6-a3cd177658e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.220402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daefee1-146b-442d-80d6-a3cd177658e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.220667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b3037-4655-469a-8464-a654d4e87c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.220833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daefee1-146b-442d-80d6-a3cd177658e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.221120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daefee1-146b-442d-80d6-a3cd177658e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.221512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b3037-4655-469a-8464-a654d4e87c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.222365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b3037-4655-469a-8464-a654d4e87c54 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.222541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daefee1-146b-442d-80d6-a3cd177658e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.223516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b3037-4655-469a-8464-a654d4e87c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.224729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daefee1-146b-442d-80d6-a3cd177658e2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.225695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b3037-4655-469a-8464-a654d4e87c54 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.255473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 795/1581] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.257108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774428e-36f4-4265-897b-e81223524295 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.257403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774428e-36f4-4265-897b-e81223524295 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.257686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774428e-36f4-4265-897b-e81223524295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.259123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 787/1582] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.260081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daefee1-146b-442d-80d6-a3cd177658e2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.297266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c774428e-36f4-4265-897b-e81223524295 None None] MFA Rules not processed for user `829c8ebd4eaa4b47b130ce6c2a8f80e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:00.300385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c774428e-36f4-4265-897b-e81223524295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:00.353463 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daefee1-146b-442d-80d6-a3cd177658e2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.358800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2922076-a707-4e3d-9678-f76abc66c4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.359708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2922076-a707-4e3d-9678-f76abc66c4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.360259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2922076-a707-4e3d-9678-f76abc66c4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.360692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2922076-a707-4e3d-9678-f76abc66c4e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.361578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2922076-a707-4e3d-9678-f76abc66c4e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.364189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2922076-a707-4e3d-9678-f76abc66c4e4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.399247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 796/1583] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:00.399247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774428e-36f4-4265-897b-e81223524295 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.399570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 788/1584] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.399570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2922076-a707-4e3d-9678-f76abc66c4e4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.411195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774428e-36f4-4265-897b-e81223524295 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.416051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2922076-a707-4e3d-9678-f76abc66c4e4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.416740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662789-cfde-45b8-91a3-7921216e8107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.417790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662789-cfde-45b8-91a3-7921216e8107 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.418078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662789-cfde-45b8-91a3-7921216e8107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.418452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c662789-cfde-45b8-91a3-7921216e8107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.419297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c662789-cfde-45b8-91a3-7921216e8107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.419987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc965f1-0ec1-46f4-9ef4-1b4ab0618567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.420835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc965f1-0ec1-46f4-9ef4-1b4ab0618567 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.421042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c662789-cfde-45b8-91a3-7921216e8107 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.421165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc965f1-0ec1-46f4-9ef4-1b4ab0618567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.421549 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc965f1-0ec1-46f4-9ef4-1b4ab0618567 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.422352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc965f1-0ec1-46f4-9ef4-1b4ab0618567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.425342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc965f1-0ec1-46f4-9ef4-1b4ab0618567 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.437202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 797/1585] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:00.438437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662789-cfde-45b8-91a3-7921216e8107 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.451423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c662789-cfde-45b8-91a3-7921216e8107 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.452544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 789/1586] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/users => generated 290 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.453460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc965f1-0ec1-46f4-9ef4-1b4ab0618567 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.461170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080beae3-209d-48c5-8dbc-05f5be4b408e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.461648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080beae3-209d-48c5-8dbc-05f5be4b408e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.461989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080beae3-209d-48c5-8dbc-05f5be4b408e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.462278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080beae3-209d-48c5-8dbc-05f5be4b408e admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.462976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080beae3-209d-48c5-8dbc-05f5be4b408e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.465450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080beae3-209d-48c5-8dbc-05f5be4b408e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.468783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc965f1-0ec1-46f4-9ef4-1b4ab0618567 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.469242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:00.475331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c878b-0d31-478c-9a93-7232c2eadf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.476135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c878b-0d31-478c-9a93-7232c2eadf88 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.476522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c878b-0d31-478c-9a93-7232c2eadf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.477052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c878b-0d31-478c-9a93-7232c2eadf88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.479179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c878b-0d31-478c-9a93-7232c2eadf88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.481254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c878b-0d31-478c-9a93-7232c2eadf88 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.502037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 790/1587] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:00.503240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c878b-0d31-478c-9a93-7232c2eadf88 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.504422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 798/1588] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.505437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080beae3-209d-48c5-8dbc-05f5be4b408e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.524757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c878b-0d31-478c-9a93-7232c2eadf88 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.590555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080beae3-209d-48c5-8dbc-05f5be4b408e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.596313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca82613-7b7f-4e17-a14a-8f634f1a85c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.597157 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca82613-7b7f-4e17-a14a-8f634f1a85c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.597662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca82613-7b7f-4e17-a14a-8f634f1a85c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.598122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca82613-7b7f-4e17-a14a-8f634f1a85c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.598907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca82613-7b7f-4e17-a14a-8f634f1a85c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.601491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca82613-7b7f-4e17-a14a-8f634f1a85c6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.613757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f998adc-9777-481a-ae7a-764335e2b5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.617745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f998adc-9777-481a-ae7a-764335e2b5a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.618245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f998adc-9777-481a-ae7a-764335e2b5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.618613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f998adc-9777-481a-ae7a-764335e2b5a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.619390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f998adc-9777-481a-ae7a-764335e2b5a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.621984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f998adc-9777-481a-ae7a-764335e2b5a5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.639083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 799/1589] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.642572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca82613-7b7f-4e17-a14a-8f634f1a85c6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.652897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 791/1590] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.654316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f998adc-9777-481a-ae7a-764335e2b5a5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.664873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:00.672109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f998adc-9777-481a-ae7a-764335e2b5a5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.763228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca82613-7b7f-4e17-a14a-8f634f1a85c6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.774941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082cb79-0b3b-434a-b889-39955a6da369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.775638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082cb79-0b3b-434a-b889-39955a6da369 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.775928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082cb79-0b3b-434a-b889-39955a6da369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.776240 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082cb79-0b3b-434a-b889-39955a6da369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.776998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082cb79-0b3b-434a-b889-39955a6da369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.778845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082cb79-0b3b-434a-b889-39955a6da369 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.791981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664e5ea-4a1b-44cc-a833-a60b29412f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.792865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 800/1591] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:00.792983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664e5ea-4a1b-44cc-a833-a60b29412f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.792983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664e5ea-4a1b-44cc-a833-a60b29412f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.793356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664e5ea-4a1b-44cc-a833-a60b29412f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.794000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664e5ea-4a1b-44cc-a833-a60b29412f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.795800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664e5ea-4a1b-44cc-a833-a60b29412f00 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.797012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082cb79-0b3b-434a-b889-39955a6da369 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.811830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 792/1592] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:00.811830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664e5ea-4a1b-44cc-a833-a60b29412f00 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.812372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082cb79-0b3b-434a-b889-39955a6da369 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.818592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e116d1e-b3be-4496-a810-3ab0dfbe65d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.819281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e116d1e-b3be-4496-a810-3ab0dfbe65d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.819570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e116d1e-b3be-4496-a810-3ab0dfbe65d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.819905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e116d1e-b3be-4496-a810-3ab0dfbe65d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.820634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e116d1e-b3be-4496-a810-3ab0dfbe65d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.821814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664e5ea-4a1b-44cc-a833-a60b29412f00 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.822272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e116d1e-b3be-4496-a810-3ab0dfbe65d3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.828631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7392a04-47fe-4afa-9a34-a6042b8aecf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.829549 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7392a04-47fe-4afa-9a34-a6042b8aecf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.829680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7392a04-47fe-4afa-9a34-a6042b8aecf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.829844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7392a04-47fe-4afa-9a34-a6042b8aecf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.830570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7392a04-47fe-4afa-9a34-a6042b8aecf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.832838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7392a04-47fe-4afa-9a34-a6042b8aecf5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.836108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 801/1593] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:00.836407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e116d1e-b3be-4496-a810-3ab0dfbe65d3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.852234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 793/1594] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:00.853037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7392a04-47fe-4afa-9a34-a6042b8aecf5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.861068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e116d1e-b3be-4496-a810-3ab0dfbe65d3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.866724 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7392a04-47fe-4afa-9a34-a6042b8aecf5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.872286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acb419-6010-4bc6-a73a-d0bcdb4dc82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.872986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acb419-6010-4bc6-a73a-d0bcdb4dc82b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.873280 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acb419-6010-4bc6-a73a-d0bcdb4dc82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.873553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acb419-6010-4bc6-a73a-d0bcdb4dc82b admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.874256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acb419-6010-4bc6-a73a-d0bcdb4dc82b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.876334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3acb419-6010-4bc6-a73a-d0bcdb4dc82b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.876893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54294b44-2c9d-4362-91db-cb45c0d3c5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.878753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54294b44-2c9d-4362-91db-cb45c0d3c5cc admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.879032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54294b44-2c9d-4362-91db-cb45c0d3c5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.879362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54294b44-2c9d-4362-91db-cb45c0d3c5cc admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.880079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54294b44-2c9d-4362-91db-cb45c0d3c5cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.882558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54294b44-2c9d-4362-91db-cb45c0d3c5cc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.906738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 794/1595] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/users => generated 296 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.906738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54294b44-2c9d-4362-91db-cb45c0d3c5cc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.911203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 802/1596] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:00 2022] POST /identity/v3/users => generated 294 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:00.912117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acb419-6010-4bc6-a73a-d0bcdb4dc82b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.922349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54294b44-2c9d-4362-91db-cb45c0d3c5cc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.923673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acb419-6010-4bc6-a73a-d0bcdb4dc82b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.929404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19769d5-edb1-4494-8097-d0e0283d4eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.930234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19769d5-edb1-4494-8097-d0e0283d4eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.930586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19769d5-edb1-4494-8097-d0e0283d4eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.930935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19769d5-edb1-4494-8097-d0e0283d4eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.931055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf67fa3-801e-404b-a60e-a64dc9cbb52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.931975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf67fa3-801e-404b-a60e-a64dc9cbb52f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.932096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19769d5-edb1-4494-8097-d0e0283d4eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.932604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf67fa3-801e-404b-a60e-a64dc9cbb52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.933259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf67fa3-801e-404b-a60e-a64dc9cbb52f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.933902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19769d5-edb1-4494-8097-d0e0283d4eb4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.934423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf67fa3-801e-404b-a60e-a64dc9cbb52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.937413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf67fa3-801e-404b-a60e-a64dc9cbb52f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:00.948148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 795/1597] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:00.949091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19769d5-edb1-4494-8097-d0e0283d4eb4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.957798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 803/1598] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:00.957798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf67fa3-801e-404b-a60e-a64dc9cbb52f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:00.965260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19769d5-edb1-4494-8097-d0e0283d4eb4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.972384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf67fa3-801e-404b-a60e-a64dc9cbb52f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:00.972800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21999f-01f5-4b8b-ac04-443924cbc477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.972800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21999f-01f5-4b8b-ac04-443924cbc477 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.972800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21999f-01f5-4b8b-ac04-443924cbc477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.972800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21999f-01f5-4b8b-ac04-443924cbc477 admin admin] PATH_INFO: `/v3/projects/af0e2e127e9941cc8118d8f77f419f64/users/949c0230b6304516a2c22abea205bc43/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.983581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a74e19-c025-47ae-af80-889d41946197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:00.984320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a74e19-c025-47ae-af80-889d41946197 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:00.984673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a74e19-c025-47ae-af80-889d41946197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:00.985034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a74e19-c025-47ae-af80-889d41946197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:00.985862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a74e19-c025-47ae-af80-889d41946197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:00.988578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a74e19-c025-47ae-af80-889d41946197 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.022366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21999f-01f5-4b8b-ac04-443924cbc477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0e2e127e9941cc8118d8f77f419f64, user_id=949c0230b6304516a2c22abea205bc43, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.023688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 804/1599] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:01.025442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21999f-01f5-4b8b-ac04-443924cbc477 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.029602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a74e19-c025-47ae-af80-889d41946197 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.045979 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a74e19-c025-47ae-af80-889d41946197 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.067742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:01.070279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 796/1600] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:00 2022] PUT /identity/v3/projects/af0e2e127e9941cc8118d8f77f419f64/users/949c0230b6304516a2c22abea205bc43/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:01.071246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21999f-01f5-4b8b-ac04-443924cbc477 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.159823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.160849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.161132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.161411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] PATH_INFO: `/v3/projects/c988db8e79bf4c98ac288105177f90f2/users/166b42a5d6c04659afb20c088dfead76/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.188118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c988db8e79bf4c98ac288105177f90f2, user_id=166b42a5d6c04659afb20c088dfead76, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.189298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:04:01.189298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 30 12:04:01.189298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: May 30 12:04:01.191767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.211720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21999f-01f5-4b8b-ac04-443924cbc477 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.220961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 805/1601] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:01 2022] PUT /identity/v3/projects/c988db8e79bf4c98ac288105177f90f2/users/166b42a5d6c04659afb20c088dfead76/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:01.220961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.266333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:01.287047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4981ede-30b8-4ac2-a998-b61573cec6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.287941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4981ede-30b8-4ac2-a998-b61573cec6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.288329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4981ede-30b8-4ac2-a998-b61573cec6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.288670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4981ede-30b8-4ac2-a998-b61573cec6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.289621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4981ede-30b8-4ac2-a998-b61573cec6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.291792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4981ede-30b8-4ac2-a998-b61573cec6ec admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.305559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94718f4-cd2e-444a-9084-a4fc68d15622 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.307859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 797/1602] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:01.309374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4981ede-30b8-4ac2-a998-b61573cec6ec admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.314824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0da34-83c9-4114-996a-04eefadf2028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.316224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0da34-83c9-4114-996a-04eefadf2028 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.316547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0da34-83c9-4114-996a-04eefadf2028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.316942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0da34-83c9-4114-996a-04eefadf2028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.317751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0da34-83c9-4114-996a-04eefadf2028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.319389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0da34-83c9-4114-996a-04eefadf2028 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.322163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4981ede-30b8-4ac2-a998-b61573cec6ec admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.327392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5de5a4-77f5-45b2-aac8-331435993884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.328495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de5a4-77f5-45b2-aac8-331435993884 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.328996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de5a4-77f5-45b2-aac8-331435993884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.329493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de5a4-77f5-45b2-aac8-331435993884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.330417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5de5a4-77f5-45b2-aac8-331435993884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.330616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 806/1603] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:01.334217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0da34-83c9-4114-996a-04eefadf2028 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.336243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5de5a4-77f5-45b2-aac8-331435993884 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.342313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0da34-83c9-4114-996a-04eefadf2028 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.348218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9aa75-f0a8-409e-b34d-62432f8ba788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.348752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 798/1604] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:01.349124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9aa75-f0a8-409e-b34d-62432f8ba788 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.349358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5de5a4-77f5-45b2-aac8-331435993884 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.349441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9aa75-f0a8-409e-b34d-62432f8ba788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.349707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9aa75-f0a8-409e-b34d-62432f8ba788 admin admin] PATH_INFO: `/v3/projects/20763921c37c4e81bba702e09bf628b2/users/7575179edbda45208561c983fc5b956c/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.359368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5de5a4-77f5-45b2-aac8-331435993884 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.365211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad56da-bf38-4529-bf3f-d85616d0af0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.366117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad56da-bf38-4529-bf3f-d85616d0af0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.366396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad56da-bf38-4529-bf3f-d85616d0af0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.366672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad56da-bf38-4529-bf3f-d85616d0af0c admin admin] PATH_INFO: `/v3/projects/a60b241805384ef49352b3791cfd3d8c/users/e591526a8c3645de9c5965a40f8ef90a/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.367471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9aa75-f0a8-409e-b34d-62432f8ba788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20763921c37c4e81bba702e09bf628b2, user_id=7575179edbda45208561c983fc5b956c, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.370998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9aa75-f0a8-409e-b34d-62432f8ba788 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.386251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad56da-bf38-4529-bf3f-d85616d0af0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60b241805384ef49352b3791cfd3d8c, user_id=e591526a8c3645de9c5965a40f8ef90a, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.388667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad56da-bf38-4529-bf3f-d85616d0af0c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.388871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 807/1605] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:01 2022] PUT /identity/v3/projects/20763921c37c4e81bba702e09bf628b2/users/7575179edbda45208561c983fc5b956c/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:01.394216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd17fb1-07ac-4f77-8dd8-f6545d8e3fdb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.394509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd17fb1-07ac-4f77-8dd8-f6545d8e3fdb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.394764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd17fb1-07ac-4f77-8dd8-f6545d8e3fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.406703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 799/1606] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:01 2022] PUT /identity/v3/projects/a60b241805384ef49352b3791cfd3d8c/users/e591526a8c3645de9c5965a40f8ef90a/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:01.408449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d8e7-b70a-4598-8d75-d73910e8b5dd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.408849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d8e7-b70a-4598-8d75-d73910e8b5dd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.410256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d8e7-b70a-4598-8d75-d73910e8b5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.427126 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2fd17fb1-07ac-4f77-8dd8-f6545d8e3fdb None None] MFA Rules not processed for user `949c0230b6304516a2c22abea205bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:01.428138 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-32c7d8e7-b70a-4598-8d75-d73910e8b5dd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:01.428361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2fd17fb1-07ac-4f77-8dd8-f6545d8e3fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:01.429527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-32c7d8e7-b70a-4598-8d75-d73910e8b5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:01.501754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:01.505454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 800/1607] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:01.506278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7d8e7-b70a-4598-8d75-d73910e8b5dd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.507483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 808/1608] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:01.509260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd17fb1-07ac-4f77-8dd8-f6545d8e3fdb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.521197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7d8e7-b70a-4598-8d75-d73910e8b5dd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.527178 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871c3bd-73a6-49f5-9b5b-bafd678c9d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.528305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871c3bd-73a6-49f5-9b5b-bafd678c9d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.528677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871c3bd-73a6-49f5-9b5b-bafd678c9d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.529025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871c3bd-73a6-49f5-9b5b-bafd678c9d33 admin admin] PATH_INFO: `/v3/projects/de1af30de2034b34bbf58ccbb5ffd845/users/a7ced8e0f2cb4582ba4daccabfd6670b/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.529142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd17fb1-07ac-4f77-8dd8-f6545d8e3fdb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.538417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b7367-9eb0-483c-9be8-0963409e7688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.539109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b7367-9eb0-483c-9be8-0963409e7688 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.539396 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b7367-9eb0-483c-9be8-0963409e7688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.539677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b7367-9eb0-483c-9be8-0963409e7688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.540449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b7367-9eb0-483c-9be8-0963409e7688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.542080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b7367-9eb0-483c-9be8-0963409e7688 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.547947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871c3bd-73a6-49f5-9b5b-bafd678c9d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1af30de2034b34bbf58ccbb5ffd845, user_id=a7ced8e0f2cb4582ba4daccabfd6670b, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.551092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871c3bd-73a6-49f5-9b5b-bafd678c9d33 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.559273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 809/1609] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:01.559273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b7367-9eb0-483c-9be8-0963409e7688 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.575028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 801/1610] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:01 2022] PUT /identity/v3/projects/de1af30de2034b34bbf58ccbb5ffd845/users/a7ced8e0f2cb4582ba4daccabfd6670b/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:01.577395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0aa0b-cf0c-4959-b544-954198cb5311 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.577966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0aa0b-cf0c-4959-b544-954198cb5311 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.578531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0aa0b-cf0c-4959-b544-954198cb5311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.581487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b7367-9eb0-483c-9be8-0963409e7688 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.605999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bea0aa0b-cf0c-4959-b544-954198cb5311 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:01.607603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bea0aa0b-cf0c-4959-b544-954198cb5311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:01.681141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590457ef-e72c-4103-99d5-42c24b6bd385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.682064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590457ef-e72c-4103-99d5-42c24b6bd385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.682352 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590457ef-e72c-4103-99d5-42c24b6bd385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.683742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590457ef-e72c-4103-99d5-42c24b6bd385 admin admin] PATH_INFO: `/v3/projects/8e98cc41e8064c90b0e45b32e2d78493/users/bc47c5bf77894aefa6ecde115ed9f88f/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.697396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 802/1611] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:01.699174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14328807-5302-4f07-b774-3d5554e7888e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.699542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14328807-5302-4f07-b774-3d5554e7888e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.699973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14328807-5302-4f07-b774-3d5554e7888e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.719135 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590457ef-e72c-4103-99d5-42c24b6bd385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e98cc41e8064c90b0e45b32e2d78493, user_id=bc47c5bf77894aefa6ecde115ed9f88f, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.723494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590457ef-e72c-4103-99d5-42c24b6bd385 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.723681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-14328807-5302-4f07-b774-3d5554e7888e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:01.725337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-14328807-5302-4f07-b774-3d5554e7888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:01.735969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 803/1612] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:01.737126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14328807-5302-4f07-b774-3d5554e7888e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.739390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-14328807-5302-4f07-b774-3d5554e7888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:01.743345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 810/1613] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:01 2022] PUT /identity/v3/projects/8e98cc41e8064c90b0e45b32e2d78493/users/bc47c5bf77894aefa6ecde115ed9f88f/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:01.745147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667ed9-43d0-48d6-a7b6-1926c7b00f7b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.745684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667ed9-43d0-48d6-a7b6-1926c7b00f7b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.746141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa667ed9-43d0-48d6-a7b6-1926c7b00f7b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.748297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 811/1614] 10.222.0.32 () {54 vars in 961 bytes} [Mon May 30 12:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:04:01.749555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa667ed9-43d0-48d6-a7b6-1926c7b00f7b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.765385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:01.819455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14328807-5302-4f07-b774-3d5554e7888e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.854550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa667ed9-43d0-48d6-a7b6-1926c7b00f7b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:01.861497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298afbd-6409-45c8-91c1-5cd17eebb149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.863487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298afbd-6409-45c8-91c1-5cd17eebb149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.863922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298afbd-6409-45c8-91c1-5cd17eebb149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.864435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298afbd-6409-45c8-91c1-5cd17eebb149 admin admin] PATH_INFO: `/v3/projects/be94251de77f443587ac2bf31be95c9d/users/2a78f3344ff0472a970a34448e55c1fc/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.884803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298afbd-6409-45c8-91c1-5cd17eebb149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be94251de77f443587ac2bf31be95c9d, user_id=2a78f3344ff0472a970a34448e55c1fc, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.887843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298afbd-6409-45c8-91c1-5cd17eebb149 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.907785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 812/1615] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:01 2022] PUT /identity/v3/projects/be94251de77f443587ac2bf31be95c9d/users/2a78f3344ff0472a970a34448e55c1fc/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:01.908766 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298afbd-6409-45c8-91c1-5cd17eebb149 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.937557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58d3cc-7297-4da7-aaa2-2adf1733cf9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:01.938255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58d3cc-7297-4da7-aaa2-2adf1733cf9d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:01.938589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58d3cc-7297-4da7-aaa2-2adf1733cf9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:01.938874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58d3cc-7297-4da7-aaa2-2adf1733cf9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:01.952763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58d3cc-7297-4da7-aaa2-2adf1733cf9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:01.954779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58d3cc-7297-4da7-aaa2-2adf1733cf9d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:01.974045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 804/1616] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:01.975111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58d3cc-7297-4da7-aaa2-2adf1733cf9d service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:01.986991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58d3cc-7297-4da7-aaa2-2adf1733cf9d service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:02.002374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:02.009670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298afbd-6409-45c8-91c1-5cd17eebb149 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:02.019104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82389056-c544-48b1-8cc9-53142d34f336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:02.020192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82389056-c544-48b1-8cc9-53142d34f336 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.020480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82389056-c544-48b1-8cc9-53142d34f336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.020758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82389056-c544-48b1-8cc9-53142d34f336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.022147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82389056-c544-48b1-8cc9-53142d34f336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:02.023894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82389056-c544-48b1-8cc9-53142d34f336 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:02.036757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 813/1617] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:02.037676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82389056-c544-48b1-8cc9-53142d34f336 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:02.073546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f5cac-7112-44f0-89a9-c00b53f57ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:02.076061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f5cac-7112-44f0-89a9-c00b53f57ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.076547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f5cac-7112-44f0-89a9-c00b53f57ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.076985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f5cac-7112-44f0-89a9-c00b53f57ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.091690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f5cac-7112-44f0-89a9-c00b53f57ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:02.094027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f5cac-7112-44f0-89a9-c00b53f57ecd service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:02.109370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 805/1618] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:02.110996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc3b6-2a67-48a5-83d3-b3c47e92680f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.111454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc3b6-2a67-48a5-83d3-b3c47e92680f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.112071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc3b6-2a67-48a5-83d3-b3c47e92680f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.123086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82389056-c544-48b1-8cc9-53142d34f336 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:02.127230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a1b57-11e9-4189-9db4-9460627597de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:02.129517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1b57-11e9-4189-9db4-9460627597de service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.129517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1b57-11e9-4189-9db4-9460627597de service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.129517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1b57-11e9-4189-9db4-9460627597de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.135502 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ed4fc3b6-2a67-48a5-83d3-b3c47e92680f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:02.137636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed4fc3b6-2a67-48a5-83d3-b3c47e92680f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:02.141250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a1b57-11e9-4189-9db4-9460627597de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:02.143242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a1b57-11e9-4189-9db4-9460627597de service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:02.149105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 806/1619] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:02.150630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60129ec-868b-4515-bcaa-c5d88dd204fb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.151090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60129ec-868b-4515-bcaa-c5d88dd204fb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.151520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60129ec-868b-4515-bcaa-c5d88dd204fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.171950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 814/1620] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:02.176628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e34690-9640-441d-87b6-0d68ae04f13e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.176628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e34690-9640-441d-87b6-0d68ae04f13e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.177759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e34690-9640-441d-87b6-0d68ae04f13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.182814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a60129ec-868b-4515-bcaa-c5d88dd204fb None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:02.186556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a60129ec-868b-4515-bcaa-c5d88dd204fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:02.266156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e5e34690-9640-441d-87b6-0d68ae04f13e None None] MFA Rules not processed for user `2a78f3344ff0472a970a34448e55c1fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:02.267411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e5e34690-9640-441d-87b6-0d68ae04f13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:02.292213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 807/1621] 10.222.0.32 () {58 vars in 1058 bytes} [Mon May 30 12:04:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:02.293177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60129ec-868b-4515-bcaa-c5d88dd204fb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:02.307348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:02.325505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60129ec-868b-4515-bcaa-c5d88dd204fb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:02.331073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872f9b6-ff87-4451-ae51-1dab09407dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:02.333018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872f9b6-ff87-4451-ae51-1dab09407dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.333370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872f9b6-ff87-4451-ae51-1dab09407dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.333716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872f9b6-ff87-4451-ae51-1dab09407dcc admin admin] PATH_INFO: `/v3/projects/de1af30de2034b34bbf58ccbb5ffd845/users/a7ced8e0f2cb4582ba4daccabfd6670b/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.335900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872f9b6-ff87-4451-ae51-1dab09407dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1af30de2034b34bbf58ccbb5ffd845, user_id=a7ced8e0f2cb4582ba4daccabfd6670b, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:02.339378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872f9b6-ff87-4451-ae51-1dab09407dcc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:02.366419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 808/1622] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:02 2022] PUT /identity/v3/projects/de1af30de2034b34bbf58ccbb5ffd845/users/a7ced8e0f2cb4582ba4daccabfd6670b/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:02.367046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872f9b6-ff87-4451-ae51-1dab09407dcc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:02.448202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 815/1623] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:02.449003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e34690-9640-441d-87b6-0d68ae04f13e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:02.473320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872f9b6-ff87-4451-ae51-1dab09407dcc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:02.476641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e34690-9640-441d-87b6-0d68ae04f13e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:02.480214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26a280-4f8d-4cce-89aa-733c481ac689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:02.481591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a280-4f8d-4cce-89aa-733c481ac689 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.481975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a280-4f8d-4cce-89aa-733c481ac689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.482331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a280-4f8d-4cce-89aa-733c481ac689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.578133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26a280-4f8d-4cce-89aa-733c481ac689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:02.580105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26a280-4f8d-4cce-89aa-733c481ac689 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:02.597627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 809/1624] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:02.599319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e4b93-4901-4a87-9e22-cd6dc15a3781 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.599684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e4b93-4901-4a87-9e22-cd6dc15a3781 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.600040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e4b93-4901-4a87-9e22-cd6dc15a3781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.612977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c39fef-afc0-46ff-85f4-4d5defd352bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:02.613719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39fef-afc0-46ff-85f4-4d5defd352bc service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.614071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39fef-afc0-46ff-85f4-4d5defd352bc service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.614426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39fef-afc0-46ff-85f4-4d5defd352bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.654866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-342e4b93-4901-4a87-9e22-cd6dc15a3781 None None] MFA Rules not processed for user `a7ced8e0f2cb4582ba4daccabfd6670b`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:02.656281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-342e4b93-4901-4a87-9e22-cd6dc15a3781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:02.663676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:02.722594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c39fef-afc0-46ff-85f4-4d5defd352bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:02.725051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c39fef-afc0-46ff-85f4-4d5defd352bc service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:02.755898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 816/1625] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:02.757331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c39fef-afc0-46ff-85f4-4d5defd352bc service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:02.780600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 810/1626] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:02.791731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e4b93-4901-4a87-9e22-cd6dc15a3781 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:02.809353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c39fef-afc0-46ff-85f4-4d5defd352bc service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:02.831363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dd879-971b-4bb6-854d-e0749d14f06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:02.833244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:02.834383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dd879-971b-4bb6-854d-e0749d14f06b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:02.838898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dd879-971b-4bb6-854d-e0749d14f06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:02.839842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dd879-971b-4bb6-854d-e0749d14f06b admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:02.844916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dd879-971b-4bb6-854d-e0749d14f06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:02.855901 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dd879-971b-4bb6-854d-e0749d14f06b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:02.927376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e4b93-4901-4a87-9e22-cd6dc15a3781 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:02.928433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 817/1627] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:02 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:02.933170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dd879-971b-4bb6-854d-e0749d14f06b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:03.063512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad235dc-d60b-4e77-8992-2ba228847fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:03.064218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad235dc-d60b-4e77-8992-2ba228847fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.065088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dd879-971b-4bb6-854d-e0749d14f06b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:03.065213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad235dc-d60b-4e77-8992-2ba228847fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.066848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad235dc-d60b-4e77-8992-2ba228847fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.082077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c740-6116-4eee-b223-f7b942d0f258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:03.082956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c740-6116-4eee-b223-f7b942d0f258 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.083347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c740-6116-4eee-b223-f7b942d0f258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.083636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c740-6116-4eee-b223-f7b942d0f258 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.084424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3c740-6116-4eee-b223-f7b942d0f258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:03.090697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3c740-6116-4eee-b223-f7b942d0f258 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:03.135072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 818/1628] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:02 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:04:03.138938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c740-6116-4eee-b223-f7b942d0f258 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:03.157866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c740-6116-4eee-b223-f7b942d0f258 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:03.169041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56597f4-d2dd-4867-aa64-730ba9d44cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:03.169736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56597f4-d2dd-4867-aa64-730ba9d44cea admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.170022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56597f4-d2dd-4867-aa64-730ba9d44cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.170298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56597f4-d2dd-4867-aa64-730ba9d44cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.171030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56597f4-d2dd-4867-aa64-730ba9d44cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:03.172684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56597f4-d2dd-4867-aa64-730ba9d44cea admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:03.204968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 819/1629] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:03.206100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56597f4-d2dd-4867-aa64-730ba9d44cea admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:03.231803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56597f4-d2dd-4867-aa64-730ba9d44cea admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:03.245014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57219d4c-992f-436a-af32-8f127b989884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:03.246106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57219d4c-992f-436a-af32-8f127b989884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.246397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57219d4c-992f-436a-af32-8f127b989884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.246673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57219d4c-992f-436a-af32-8f127b989884 admin admin] PATH_INFO: `/v3/projects/b5b4fa4be5664ba98e415b55fd7448e4/users/34b2958740d641458907654533a8ea52/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.247711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad235dc-d60b-4e77-8992-2ba228847fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:03.254811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad235dc-d60b-4e77-8992-2ba228847fd1 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:03.277009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57219d4c-992f-436a-af32-8f127b989884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b4fa4be5664ba98e415b55fd7448e4, user_id=34b2958740d641458907654533a8ea52, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:03.279740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57219d4c-992f-436a-af32-8f127b989884 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:03.287730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 811/1630] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:03.304752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 820/1631] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:03 2022] PUT /identity/v3/projects/b5b4fa4be5664ba98e415b55fd7448e4/users/34b2958740d641458907654533a8ea52/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:03.309315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad235dc-d60b-4e77-8992-2ba228847fd1 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:03.330413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad235dc-d60b-4e77-8992-2ba228847fd1 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:03.455393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57219d4c-992f-436a-af32-8f127b989884 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:03.510931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5e3c4-a441-4f7d-934c-6a5ef86e7027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:03.512906 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e3c4-a441-4f7d-934c-6a5ef86e7027 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.514821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e3c4-a441-4f7d-934c-6a5ef86e7027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.515252 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e3c4-a441-4f7d-934c-6a5ef86e7027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.519430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5e3c4-a441-4f7d-934c-6a5ef86e7027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:03.525521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5e3c4-a441-4f7d-934c-6a5ef86e7027 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:03.555717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 812/1632] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:03.560012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5e3c4-a441-4f7d-934c-6a5ef86e7027 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:03.605070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5e3c4-a441-4f7d-934c-6a5ef86e7027 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:03.622570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd760b0-7c51-4de4-b74c-1ffac0a2ea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:03.626777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd760b0-7c51-4de4-b74c-1ffac0a2ea5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.628289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd760b0-7c51-4de4-b74c-1ffac0a2ea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.629696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd760b0-7c51-4de4-b74c-1ffac0a2ea5c admin admin] PATH_INFO: `/v3/projects/b5b4fa4be5664ba98e415b55fd7448e4/users/34b2958740d641458907654533a8ea52/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.636261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd760b0-7c51-4de4-b74c-1ffac0a2ea5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b4fa4be5664ba98e415b55fd7448e4, user_id=34b2958740d641458907654533a8ea52, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:03.644750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57219d4c-992f-436a-af32-8f127b989884 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:03.649710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd760b0-7c51-4de4-b74c-1ffac0a2ea5c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:03.656623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f256ec2-3f48-48e0-a264-0d076944a7be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:03.658738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256ec2-3f48-48e0-a264-0d076944a7be service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.658738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256ec2-3f48-48e0-a264-0d076944a7be service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.660864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256ec2-3f48-48e0-a264-0d076944a7be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.672346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:03.686144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 813/1633] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:03 2022] PUT /identity/v3/projects/b5b4fa4be5664ba98e415b55fd7448e4/users/34b2958740d641458907654533a8ea52/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:03.697019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b6f0-21b8-4c80-91a5-611e48d62c6a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.697258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b6f0-21b8-4c80-91a5-611e48d62c6a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.697422 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6b6f0-21b8-4c80-91a5-611e48d62c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.763427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:03.775024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3be6b6f0-21b8-4c80-91a5-611e48d62c6a None None] MFA Rules not processed for user `34b2958740d641458907654533a8ea52`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:03.779340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3be6b6f0-21b8-4c80-91a5-611e48d62c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:03.844088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f256ec2-3f48-48e0-a264-0d076944a7be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:03.849717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f256ec2-3f48-48e0-a264-0d076944a7be service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:03.875131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 821/1634] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:03.886849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539ea1f-ef86-4a3f-98d8-47bc6ad85558 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.887517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539ea1f-ef86-4a3f-98d8-47bc6ad85558 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.888719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539ea1f-ef86-4a3f-98d8-47bc6ad85558 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.892062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 822/1635] 10.222.0.32 () {54 vars in 961 bytes} [Mon May 30 12:04:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 12:04:03.913393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e801c-3eed-445e-8691-35c28fa2a77e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:03.913740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e801c-3eed-445e-8691-35c28fa2a77e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:03.914028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e801c-3eed-445e-8691-35c28fa2a77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:03.938460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-968e801c-3eed-445e-8691-35c28fa2a77e None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:03.939995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-968e801c-3eed-445e-8691-35c28fa2a77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:03.981784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 814/1636] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:03.994476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6b6f0-21b8-4c80-91a5-611e48d62c6a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:04.145674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 823/1637] 10.222.0.32 () {58 vars in 1058 bytes} [Mon May 30 12:04:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:04.149596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6b6f0-21b8-4c80-91a5-611e48d62c6a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:04.151626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599f483-c7b7-4d12-b228-ab8fb74016ab None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:04.152205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599f483-c7b7-4d12-b228-ab8fb74016ab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:04.153386 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599f483-c7b7-4d12-b228-ab8fb74016ab None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:04.157483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 824/1638] 10.222.0.32 () {54 vars in 912 bytes} [Mon May 30 12:04:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:04:04.165697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20492ba8-52f0-4f96-ace8-e0f2ef3f6344 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:04.165990 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20492ba8-52f0-4f96-ace8-e0f2ef3f6344 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:04.166414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20492ba8-52f0-4f96-ace8-e0f2ef3f6344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:04.168215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15425f5-7d2b-432d-9297-c6e7c31290e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:04.169482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15425f5-7d2b-432d-9297-c6e7c31290e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:04.170225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15425f5-7d2b-432d-9297-c6e7c31290e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:04.171087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15425f5-7d2b-432d-9297-c6e7c31290e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:04.202465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15425f5-7d2b-432d-9297-c6e7c31290e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:04.217564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15425f5-7d2b-432d-9297-c6e7c31290e6 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:04.231368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-20492ba8-52f0-4f96-ace8-e0f2ef3f6344 None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:04.234166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-20492ba8-52f0-4f96-ace8-e0f2ef3f6344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:04.265070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 815/1639] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:04.274730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 825/1640] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:04:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:04.317187 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20492ba8-52f0-4f96-ace8-e0f2ef3f6344 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:04.319538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15425f5-7d2b-432d-9297-c6e7c31290e6 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:04.356217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20492ba8-52f0-4f96-ace8-e0f2ef3f6344 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:04.378091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4009f-3f02-484a-b27d-c1b28073bb46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:04.382533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4009f-3f02-484a-b27d-c1b28073bb46 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:04.384109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4009f-3f02-484a-b27d-c1b28073bb46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:04.385240 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4009f-3f02-484a-b27d-c1b28073bb46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:04.426172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4009f-3f02-484a-b27d-c1b28073bb46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:04.431561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4009f-3f02-484a-b27d-c1b28073bb46 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:04.462346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 826/1641] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:04.515652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15425f5-7d2b-432d-9297-c6e7c31290e6 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:04.526585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:04.529792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:04.530346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:04.531134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] PATH_INFO: `/v3/users/2a78f3344ff0472a970a34448e55c1fc` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:04.533157 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a78f3344ff0472a970a34448e55c1fc)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:04.536385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:04.661266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:04.664495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a78f3344ff0472a970a34448e55c1fc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:04.698754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a78f3344ff0472a970a34448e55c1fc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:04.700521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a78f3344ff0472a970a34448e55c1fc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:04.712998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768ac7-4786-43c0-898e-a5cf1dd268b1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:04.713294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768ac7-4786-43c0-898e-a5cf1dd268b1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:04.713581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768ac7-4786-43c0-898e-a5cf1dd268b1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:04.714977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 827/1642] 10.222.0.32 () {54 vars in 912 bytes} [Mon May 30 12:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 30 12:04:04.720569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a78f3344ff0472a970a34448e55c1fc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:04.725082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2caa8-9e34-4800-a732-b78cdcec1ecc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:04.725393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2caa8-9e34-4800-a732-b78cdcec1ecc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:04.725649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2caa8-9e34-4800-a732-b78cdcec1ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:04.729118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7c52e5b8-8e39-4f15-b482-db362cd002a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a78f3344ff0472a970a34448e55c1fc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:04.758748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 816/1643] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:04 2022] DELETE /identity/v3/users/2a78f3344ff0472a970a34448e55c1fc => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:04.760983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4fea3-3faf-4be4-a5e9-1412c927e305 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:04.761382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4fea3-3faf-4be4-a5e9-1412c927e305 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:04.761573 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4fea3-3faf-4be4-a5e9-1412c927e305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:04.766073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fca2caa8-9e34-4800-a732-b78cdcec1ecc None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:04.772779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fca2caa8-9e34-4800-a732-b78cdcec1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:04.801625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5bb4fea3-3faf-4be4-a5e9-1412c927e305 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:04.802872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5bb4fea3-3faf-4be4-a5e9-1412c927e305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:04.926388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 817/1644] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:04.937902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 828/1645] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:04:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:04.951913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4fea3-3faf-4be4-a5e9-1412c927e305 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:04.953928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5bb4fea3-3faf-4be4-a5e9-1412c927e305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:05.066008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2caa8-9e34-4800-a732-b78cdcec1ecc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:05.067761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fca2caa8-9e34-4800-a732-b78cdcec1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:05.091606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:05.113030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4fea3-3faf-4be4-a5e9-1412c927e305 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:05.126542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:05.128697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:05.128697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:05.128697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:05.152562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:05.154058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:05.195191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 818/1646] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:05.203000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2caa8-9e34-4800-a732-b78cdcec1ecc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:05.209407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5d44b-8687-44e9-b3aa-064ea900d30a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:05.210086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d44b-8687-44e9-b3aa-064ea900d30a service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:05.210375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d44b-8687-44e9-b3aa-064ea900d30a service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:05.210653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d44b-8687-44e9-b3aa-064ea900d30a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:05.220828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5d44b-8687-44e9-b3aa-064ea900d30a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:05.222864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5d44b-8687-44e9-b3aa-064ea900d30a service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:05.235601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 829/1647] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:05.434964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:05.436618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:05.462752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5ac51-2531-4d96-bd30-68fc0cf000d3 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:05.474203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:05.474203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:05.474514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:05.474923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] PATH_INFO: `/v3/users/949c0230b6304516a2c22abea205bc43` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:05.480719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=949c0230b6304516a2c22abea205bc43)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:05.483059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:05.568761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949c0230b6304516a2c22abea205bc43'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:05.594545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:05.600798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949c0230b6304516a2c22abea205bc43'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:05.601665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949c0230b6304516a2c22abea205bc43'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:05.615099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949c0230b6304516a2c22abea205bc43'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:05.616350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949c0230b6304516a2c22abea205bc43'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:05.651927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 819/1648] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:05 2022] DELETE /identity/v3/users/949c0230b6304516a2c22abea205bc43 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:05.656546 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58b61-d323-42c6-a4f6-2b0b77d7b5c2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:05.657675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58b61-d323-42c6-a4f6-2b0b77d7b5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:05.657991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b58b61-d323-42c6-a4f6-2b0b77d7b5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:05.709735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-16b58b61-d323-42c6-a4f6-2b0b77d7b5c2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:05.709735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-16b58b61-d323-42c6-a4f6-2b0b77d7b5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:05.860257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 830/1649] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:05.910912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:05.912578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:06.041118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf0ebc-4750-4daa-b8ec-699749533e82 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:06.049572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba14e0-c3a2-415e-bf82-2f74b6fc67b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:06.050591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba14e0-c3a2-415e-bf82-2f74b6fc67b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:06.051032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba14e0-c3a2-415e-bf82-2f74b6fc67b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:06.051317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba14e0-c3a2-415e-bf82-2f74b6fc67b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:06.110521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba14e0-c3a2-415e-bf82-2f74b6fc67b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:06.119319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba14e0-c3a2-415e-bf82-2f74b6fc67b7 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:06.133221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 820/1650] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:06.976747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d71c2-34fe-4147-9d64-9f37f3bd4e63 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:06.977608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d71c2-34fe-4147-9d64-9f37f3bd4e63 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:06.978027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d71c2-34fe-4147-9d64-9f37f3bd4e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:07.051833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a10d71c2-34fe-4147-9d64-9f37f3bd4e63 None None] MFA Rules not processed for user `e591526a8c3645de9c5965a40f8ef90a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:07.054138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a10d71c2-34fe-4147-9d64-9f37f3bd4e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:07.276327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 831/1651] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:07.278270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a24bcb-ef09-404d-9ee2-a40f2cfa711c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:07.279327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a24bcb-ef09-404d-9ee2-a40f2cfa711c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:07.279763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a24bcb-ef09-404d-9ee2-a40f2cfa711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:07.313198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-55a24bcb-ef09-404d-9ee2-a40f2cfa711c None None] MFA Rules not processed for user `e591526a8c3645de9c5965a40f8ef90a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:07.317242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-55a24bcb-ef09-404d-9ee2-a40f2cfa711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:07.364051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 821/1652] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:07.367114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d71c2-34fe-4147-9d64-9f37f3bd4e63 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:07.376578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a10d71c2-34fe-4147-9d64-9f37f3bd4e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:07.390432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a24bcb-ef09-404d-9ee2-a40f2cfa711c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:07.394131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-55a24bcb-ef09-404d-9ee2-a40f2cfa711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:07.422591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d71c2-34fe-4147-9d64-9f37f3bd4e63 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:07.445681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:07.447762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:07.449064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:07.449358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] PATH_INFO: `/v3/projects/be94251de77f443587ac2bf31be95c9d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:07.451394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be94251de77f443587ac2bf31be95c9d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:07.460000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:07.586543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:07.604926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be94251de77f443587ac2bf31be95c9d', 'request_id': 'req-550076f4-1ab8-4691-8b72-f383848b97c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.619060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be94251de77f443587ac2bf31be95c9d', 'request_id': 'req-550076f4-1ab8-4691-8b72-f383848b97c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.638165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be94251de77f443587ac2bf31be95c9d', 'request_id': 'req-550076f4-1ab8-4691-8b72-f383848b97c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.654469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a24bcb-ef09-404d-9ee2-a40f2cfa711c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:07.655726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] The token cache is being invalidate because project be94251de77f443587ac2bf31be95c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:07.656203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.657096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.660051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 832/1653] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:07 2022] DELETE /identity/v3/projects/be94251de77f443587ac2bf31be95c9d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:07.661587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:07.663155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:07.811530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550076f4-1ab8-4691-8b72-f383848b97c1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:07.811959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff394d-1275-4af3-a0d8-d9d0ff683c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:07.812855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff394d-1275-4af3-a0d8-d9d0ff683c64 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:07.813250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff394d-1275-4af3-a0d8-d9d0ff683c64 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:07.814035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff394d-1275-4af3-a0d8-d9d0ff683c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:07.815688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6fff394d-1275-4af3-a0d8-d9d0ff683c64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:07.821963 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:07.823197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:07.824921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:07.825405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] PATH_INFO: `/v3/projects/af0e2e127e9941cc8118d8f77f419f64` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:07.827977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0e2e127e9941cc8118d8f77f419f64)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:07.831310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:07.834259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:07.921646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0e2e127e9941cc8118d8f77f419f64', 'request_id': 'req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.938544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0e2e127e9941cc8118d8f77f419f64', 'request_id': 'req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.957477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0e2e127e9941cc8118d8f77f419f64', 'request_id': 'req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.976487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] The token cache is being invalidate because project af0e2e127e9941cc8118d8f77f419f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:07.977164 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.979262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:07.981085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff394d-1275-4af3-a0d8-d9d0ff683c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:07.982979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff394d-1275-4af3-a0d8-d9d0ff683c64 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:07.988008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 833/1654] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:07 2022] DELETE /identity/v3/projects/af0e2e127e9941cc8118d8f77f419f64 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:07.989446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:07.991319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:08.022066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 822/1655] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:08.027360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede51c-485b-41e4-9eb1-68dfdd6a553f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.028753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede51c-485b-41e4-9eb1-68dfdd6a553f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.029529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede51c-485b-41e4-9eb1-68dfdd6a553f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.078580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1bede51c-485b-41e4-9eb1-68dfdd6a553f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:08.081142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1bede51c-485b-41e4-9eb1-68dfdd6a553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:08.133612 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:08.141088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfb474-1f73-4c7f-ba98-c03c8c90c2f5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:08.161334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:08.162254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.163112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.163439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] PATH_INFO: `/v3/users/34b2958740d641458907654533a8ea52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.165376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b2958740d641458907654533a8ea52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:08.166729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:04:08.166729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 30 12:04:08.166729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: May 30 12:04:08.167982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:08.275490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 823/1656] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:08.278293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ae0a-11d7-4da3-9bf3-c0e7e2fee8ff None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.278893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ae0a-11d7-4da3-9bf3-c0e7e2fee8ff None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.279592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ae0a-11d7-4da3-9bf3-c0e7e2fee8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.348653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1ab6ae0a-11d7-4da3-9bf3-c0e7e2fee8ff None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:08.349934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b2958740d641458907654533a8ea52'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:08.350108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1ab6ae0a-11d7-4da3-9bf3-c0e7e2fee8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:08.360237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 824/1657] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:08.364785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6ae0a-11d7-4da3-9bf3-c0e7e2fee8ff None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:08.385354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6ae0a-11d7-4da3-9bf3-c0e7e2fee8ff None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:08.396194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:08.397036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.397445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.397794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.398626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:08.399010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:08.404394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:08.425774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 825/1658] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:08.431885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:08.445016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0debf-311e-4e5a-be95-4fb9c2a2196d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:08.448035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b2958740d641458907654533a8ea52'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:08.448896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b2958740d641458907654533a8ea52'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:08.449725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b2958740d641458907654533a8ea52'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:08.452507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b4d1-f830-447e-9f0e-b3fcf979e1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:08.453564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b4d1-f830-447e-9f0e-b3fcf979e1aa admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.454042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b4d1-f830-447e-9f0e-b3fcf979e1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.454476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b4d1-f830-447e-9f0e-b3fcf979e1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.455346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2b4d1-f830-447e-9f0e-b3fcf979e1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:08.458600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2b4d1-f830-447e-9f0e-b3fcf979e1aa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:08.475363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b2958740d641458907654533a8ea52'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:08.510001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 834/1659] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:07 2022] DELETE /identity/v3/users/34b2958740d641458907654533a8ea52 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:08.517885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 826/1660] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:08 2022] POST /identity/v3/projects => generated 375 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:08.523423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:08.526506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:08.622129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bac83-9192-47fc-b654-ea853fb77aad admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:08.726626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd6e7c-3fe3-4e77-b169-cedb0e41f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:08.727482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd6e7c-3fe3-4e77-b169-cedb0e41f7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.727785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd6e7c-3fe3-4e77-b169-cedb0e41f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.728061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd6e7c-3fe3-4e77-b169-cedb0e41f7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.728798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd6e7c-3fe3-4e77-b169-cedb0e41f7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:08.730791 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd6e7c-3fe3-4e77-b169-cedb0e41f7b7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:08.781912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 835/1661] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:08 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:08.783129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b4d1-f830-447e-9f0e-b3fcf979e1aa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:08.801911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b4d1-f830-447e-9f0e-b3fcf979e1aa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:08.819593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb0cae-4af6-48d5-8923-e8a5979cf295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:08.821949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb0cae-4af6-48d5-8923-e8a5979cf295 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.822472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb0cae-4af6-48d5-8923-e8a5979cf295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.822975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb0cae-4af6-48d5-8923-e8a5979cf295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.852042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652101-6f01-446d-a5f5-5f1108cb3ce6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.852365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652101-6f01-446d-a5f5-5f1108cb3ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.852692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652101-6f01-446d-a5f5-5f1108cb3ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.857111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb0cae-4af6-48d5-8923-e8a5979cf295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:08.859074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb0cae-4af6-48d5-8923-e8a5979cf295 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:08.889661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 827/1662] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:08.892199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb0cae-4af6-48d5-8923-e8a5979cf295 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:08.900631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:08.907978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e3652101-6f01-446d-a5f5-5f1108cb3ce6 None None] MFA Rules not processed for user `c76e8d7bfdc04ea79c83ca7fb6b9ade1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:08.911782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e3652101-6f01-446d-a5f5-5f1108cb3ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:08.912604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb0cae-4af6-48d5-8923-e8a5979cf295 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:08.921000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4fc49-abe1-4d11-a6be-71c3090befdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:08.922526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4fc49-abe1-4d11-a6be-71c3090befdf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.924941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4fc49-abe1-4d11-a6be-71c3090befdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.925491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4fc49-abe1-4d11-a6be-71c3090befdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:08.926555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4fc49-abe1-4d11-a6be-71c3090befdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:08.929461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4fc49-abe1-4d11-a6be-71c3090befdf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:08.964567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 828/1663] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:08.965789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4fc49-abe1-4d11-a6be-71c3090befdf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:08.979885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4fc49-abe1-4d11-a6be-71c3090befdf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:08.987196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007fc74-7625-428e-b107-43fe3b7a2fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:08.988390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007fc74-7625-428e-b107-43fe3b7a2fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:08.988682 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007fc74-7625-428e-b107-43fe3b7a2fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:08.988966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007fc74-7625-428e-b107-43fe3b7a2fb4 admin admin] PATH_INFO: `/v3/projects/1937b04b46c440018587024189bc0ed7/users/d1c76291c9f94809ba2210e8954343aa/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.012718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007fc74-7625-428e-b107-43fe3b7a2fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1937b04b46c440018587024189bc0ed7, user_id=d1c76291c9f94809ba2210e8954343aa, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:09.016445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007fc74-7625-428e-b107-43fe3b7a2fb4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:09.038625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 829/1664] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:08 2022] PUT /identity/v3/projects/1937b04b46c440018587024189bc0ed7/users/d1c76291c9f94809ba2210e8954343aa/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:09.050686 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa16a-273b-48d9-ad3c-c18761ff7a91 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.051600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa16a-273b-48d9-ad3c-c18761ff7a91 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.051890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa16a-273b-48d9-ad3c-c18761ff7a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.070570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 836/1665] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:09.087606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3652101-6f01-446d-a5f5-5f1108cb3ce6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:09.093347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e3652101-6f01-446d-a5f5-5f1108cb3ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:09.133656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9a5fa16a-273b-48d9-ad3c-c18761ff7a91 None None] MFA Rules not processed for user `d1c76291c9f94809ba2210e8954343aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:09.141951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9a5fa16a-273b-48d9-ad3c-c18761ff7a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:09.168968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:09.323106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3652101-6f01-446d-a5f5-5f1108cb3ce6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:09.333531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 830/1666] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:09.338436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fa16a-273b-48d9-ad3c-c18761ff7a91 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:09.466250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fa16a-273b-48d9-ad3c-c18761ff7a91 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:09.475695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed39dba-e0c7-4b21-91e7-79f2db716625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:09.477176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39dba-e0c7-4b21-91e7-79f2db716625 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.477639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39dba-e0c7-4b21-91e7-79f2db716625 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.477755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:09.478235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39dba-e0c7-4b21-91e7-79f2db716625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.478927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.479413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.479826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.480762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:09.483592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:09.534414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 831/1667] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:09 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:04:09.535520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:09.542824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:09.544144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:09.569201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39dba-e0c7-4b21-91e7-79f2db716625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:09.571238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39dba-e0c7-4b21-91e7-79f2db716625 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:09.606024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 837/1668] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:09.607096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed39dba-e0c7-4b21-91e7-79f2db716625 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:09.627132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed39dba-e0c7-4b21-91e7-79f2db716625 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:09.648583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3d576-5c03-4b04-84aa-19479797f8de admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:09.657527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:09.660014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.660472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.661635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] PATH_INFO: `/v3/projects/b5b4fa4be5664ba98e415b55fd7448e4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.663842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b4fa4be5664ba98e415b55fd7448e4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:09.665351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 30 12:04:09.665351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) May 30 12:04:09.665351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: May 30 12:04:09.667059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:09.730793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:09.746111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960afd8f-9176-4720-8f20-03ec5dcc4e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:09.747088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960afd8f-9176-4720-8f20-03ec5dcc4e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.747388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960afd8f-9176-4720-8f20-03ec5dcc4e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.747667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960afd8f-9176-4720-8f20-03ec5dcc4e62 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.748462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960afd8f-9176-4720-8f20-03ec5dcc4e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:09.750931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960afd8f-9176-4720-8f20-03ec5dcc4e62 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:09.754418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b4fa4be5664ba98e415b55fd7448e4', 'request_id': 'req-64f888c5-a580-4f43-acd8-7fe77c1eca46'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:09.778877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b4fa4be5664ba98e415b55fd7448e4', 'request_id': 'req-64f888c5-a580-4f43-acd8-7fe77c1eca46'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:09.786044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 838/1669] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:09 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:09.788672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323407a-265a-44bf-b3f6-d1c432090719 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.791780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323407a-265a-44bf-b3f6-d1c432090719 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.791905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b4fa4be5664ba98e415b55fd7448e4', 'request_id': 'req-64f888c5-a580-4f43-acd8-7fe77c1eca46'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:09.792196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323407a-265a-44bf-b3f6-d1c432090719 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.797705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 839/1670] 10.222.0.32 () {54 vars in 912 bytes} [Mon May 30 12:04:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:04:09.798614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323407a-265a-44bf-b3f6-d1c432090719 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:09.811353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] The token cache is being invalidate because project b5b4fa4be5664ba98e415b55fd7448e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:09.811978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:09.813430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-64f888c5-a580-4f43-acd8-7fe77c1eca46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:09.816245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 832/1671] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:09 2022] DELETE /identity/v3/projects/b5b4fa4be5664ba98e415b55fd7448e4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:09.818167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bdd44-2d4d-4f6f-be3c-f42bddffc762 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.819516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bdd44-2d4d-4f6f-be3c-f42bddffc762 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.819996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bdd44-2d4d-4f6f-be3c-f42bddffc762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.845910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d64bdd44-2d4d-4f6f-be3c-f42bddffc762 None None] MFA Rules not processed for user `15ea38389eee4472a8917023643be9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:09.847995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d64bdd44-2d4d-4f6f-be3c-f42bddffc762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:09.927636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323407a-265a-44bf-b3f6-d1c432090719 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:09.935980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e024d-4b9b-4e12-8f08-e06cfc8b81cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:09.937686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e024d-4b9b-4e12-8f08-e06cfc8b81cf admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.938203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e024d-4b9b-4e12-8f08-e06cfc8b81cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.938808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e024d-4b9b-4e12-8f08-e06cfc8b81cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.939784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e024d-4b9b-4e12-8f08-e06cfc8b81cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:09.943132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e024d-4b9b-4e12-8f08-e06cfc8b81cf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:09.963992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 840/1672] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:09.965456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30910c2a-df73-4491-aaee-834f0e062d40 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:09.965750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30910c2a-df73-4491-aaee-834f0e062d40 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:09.966031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30910c2a-df73-4491-aaee-834f0e062d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:09.981590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 833/1673] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:04:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:09.982405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bdd44-2d4d-4f6f-be3c-f42bddffc762 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:09.983898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d64bdd44-2d4d-4f6f-be3c-f42bddffc762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:10.001995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bdd44-2d4d-4f6f-be3c-f42bddffc762 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.009140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dbd46-0ada-4ab4-b898-904bd02eabb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.011033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dbd46-0ada-4ab4-b898-904bd02eabb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.011654 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dbd46-0ada-4ab4-b898-904bd02eabb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.012129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dbd46-0ada-4ab4-b898-904bd02eabb7 admin admin] PATH_INFO: `/v3/projects/9fa6bcce0fe6436884a274f2961b3526/users/7b2d548d83e140dbaa96792159b626e6/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.012734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-30910c2a-df73-4491-aaee-834f0e062d40 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:10.019038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-30910c2a-df73-4491-aaee-834f0e062d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:10.037557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 841/1674] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:10.037557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30910c2a-df73-4491-aaee-834f0e062d40 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.037557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-30910c2a-df73-4491-aaee-834f0e062d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:10.053911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dbd46-0ada-4ab4-b898-904bd02eabb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa6bcce0fe6436884a274f2961b3526, user_id=7b2d548d83e140dbaa96792159b626e6, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.056925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30910c2a-df73-4491-aaee-834f0e062d40 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.060275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dbd46-0ada-4ab4-b898-904bd02eabb7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.061980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc37ab2-8013-45c2-bb71-9f7d9adcc4e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.062791 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37ab2-8013-45c2-bb71-9f7d9adcc4e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.063185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37ab2-8013-45c2-bb71-9f7d9adcc4e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.063476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37ab2-8013-45c2-bb71-9f7d9adcc4e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.073921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc37ab2-8013-45c2-bb71-9f7d9adcc4e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.075650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc37ab2-8013-45c2-bb71-9f7d9adcc4e8 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.090069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 842/1675] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:10.091801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b71e1-f494-40b8-878c-1fe08c43aa97 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.092102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b71e1-f494-40b8-878c-1fe08c43aa97 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.092421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b71e1-f494-40b8-878c-1fe08c43aa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.093610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 834/1676] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:09 2022] PUT /identity/v3/projects/9fa6bcce0fe6436884a274f2961b3526/users/7b2d548d83e140dbaa96792159b626e6/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:10.095495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dbd46-0ada-4ab4-b898-904bd02eabb7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.127992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4c0b71e1-f494-40b8-878c-1fe08c43aa97 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:10.129691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c0b71e1-f494-40b8-878c-1fe08c43aa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:10.218736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:10.258783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dbd46-0ada-4ab4-b898-904bd02eabb7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.261710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 843/1677] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:10.266113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b71e1-f494-40b8-878c-1fe08c43aa97 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.274585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863be71c-2d11-411b-a852-f05c005f04fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.275291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863be71c-2d11-411b-a852-f05c005f04fc admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.275576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863be71c-2d11-411b-a852-f05c005f04fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.275852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863be71c-2d11-411b-a852-f05c005f04fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.276641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863be71c-2d11-411b-a852-f05c005f04fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.279419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863be71c-2d11-411b-a852-f05c005f04fc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.284393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b71e1-f494-40b8-878c-1fe08c43aa97 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.300972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.301867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.302698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.302698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.303395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:10.304008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.305997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.307264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 835/1678] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:10.308453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863be71c-2d11-411b-a852-f05c005f04fc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.323385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863be71c-2d11-411b-a852-f05c005f04fc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.334007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628ea5d-e071-4a75-b66e-52f7b661eb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.335549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 844/1679] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 12:04:10.336125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628ea5d-e071-4a75-b66e-52f7b661eb61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.336528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628ea5d-e071-4a75-b66e-52f7b661eb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.336930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628ea5d-e071-4a75-b66e-52f7b661eb61 admin admin] PATH_INFO: `/v3/projects/9fa6bcce0fe6436884a274f2961b3526/users/7b2d548d83e140dbaa96792159b626e6/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.339621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.340929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628ea5d-e071-4a75-b66e-52f7b661eb61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa6bcce0fe6436884a274f2961b3526, user_id=7b2d548d83e140dbaa96792159b626e6, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.348817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628ea5d-e071-4a75-b66e-52f7b661eb61 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.358717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7aa63-5cc6-4d21-b790-956c0aa32d4d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.387231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 836/1680] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:10 2022] PUT /identity/v3/projects/9fa6bcce0fe6436884a274f2961b3526/users/7b2d548d83e140dbaa96792159b626e6/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:10.389085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eb0c4-4710-45b8-8eed-22a195cf01ac None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.392761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eb0c4-4710-45b8-8eed-22a195cf01ac None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.394085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eb0c4-4710-45b8-8eed-22a195cf01ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.473219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bf7eb0c4-4710-45b8-8eed-22a195cf01ac None None] MFA Rules not processed for user `7b2d548d83e140dbaa96792159b626e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:10.476710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bf7eb0c4-4710-45b8-8eed-22a195cf01ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:10.523136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:10.525324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef106d-216e-4cb8-bb85-f251934767a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.526130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef106d-216e-4cb8-bb85-f251934767a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.526550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef106d-216e-4cb8-bb85-f251934767a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.526935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef106d-216e-4cb8-bb85-f251934767a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.527875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef106d-216e-4cb8-bb85-f251934767a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.533134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef106d-216e-4cb8-bb85-f251934767a0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.594896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 845/1681] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:10 2022] POST /identity/v3/projects => generated 387 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:10.598842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef106d-216e-4cb8-bb85-f251934767a0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.645815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 837/1682] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:10.659891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7eb0c4-4710-45b8-8eed-22a195cf01ac None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.662179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bf7eb0c4-4710-45b8-8eed-22a195cf01ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:10.755993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef106d-216e-4cb8-bb85-f251934767a0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.767966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eba2b-3dff-4568-9de7-9de825502d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.769653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eba2b-3dff-4568-9de7-9de825502d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.771718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eba2b-3dff-4568-9de7-9de825502d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.772210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eba2b-3dff-4568-9de7-9de825502d80 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.774604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eba2b-3dff-4568-9de7-9de825502d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.777708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eba2b-3dff-4568-9de7-9de825502d80 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.798354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7eb0c4-4710-45b8-8eed-22a195cf01ac None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.829752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 846/1683] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:10 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:10.831638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eba2b-3dff-4568-9de7-9de825502d80 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.853845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eba2b-3dff-4568-9de7-9de825502d80 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.866681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac214c4-e95b-4f29-9204-25c8c0859169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.867494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac214c4-e95b-4f29-9204-25c8c0859169 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.868940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac214c4-e95b-4f29-9204-25c8c0859169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.870274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac214c4-e95b-4f29-9204-25c8c0859169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.871279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac214c4-e95b-4f29-9204-25c8c0859169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.876808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac214c4-e95b-4f29-9204-25c8c0859169 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.910168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 847/1684] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:10.915330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac214c4-e95b-4f29-9204-25c8c0859169 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:10.931986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79818df-5637-4eed-8fd4-bc698494866f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.932923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79818df-5637-4eed-8fd4-bc698494866f service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.934237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79818df-5637-4eed-8fd4-bc698494866f service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.934925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac214c4-e95b-4f29-9204-25c8c0859169 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:10.935316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79818df-5637-4eed-8fd4-bc698494866f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.939241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c79818df-5637-4eed-8fd4-bc698494866f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:10.945832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab91c-4a04-4e81-81c3-0b396d1f3c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:10.946819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab91c-4a04-4e81-81c3-0b396d1f3c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:10.947272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab91c-4a04-4e81-81c3-0b396d1f3c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:10.949928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab91c-4a04-4e81-81c3-0b396d1f3c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:10.950904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ab91c-4a04-4e81-81c3-0b396d1f3c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:10.955180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ab91c-4a04-4e81-81c3-0b396d1f3c50 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:10.986419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 848/1685] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:10.987586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab91c-4a04-4e81-81c3-0b396d1f3c50 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:11.008552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab91c-4a04-4e81-81c3-0b396d1f3c50 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:11.024199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fb56d-e254-464a-adfb-293a635e1bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:11.025124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fb56d-e254-464a-adfb-293a635e1bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:11.025903 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fb56d-e254-464a-adfb-293a635e1bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:11.025903 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fb56d-e254-464a-adfb-293a635e1bc3 admin admin] PATH_INFO: `/v3/projects/c3d4bada97b241e9b2a5278f5f5365f2/users/762e83bc36254e9c84b5f9cb929d73ac/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:11.032514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79818df-5637-4eed-8fd4-bc698494866f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:11.035431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79818df-5637-4eed-8fd4-bc698494866f service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:11.046420 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 838/1686] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:11.060309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269fb56d-e254-464a-adfb-293a635e1bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d4bada97b241e9b2a5278f5f5365f2, user_id=762e83bc36254e9c84b5f9cb929d73ac, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:11.065390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269fb56d-e254-464a-adfb-293a635e1bc3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:11.088578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:11.095529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 849/1687] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:10 2022] PUT /identity/v3/projects/c3d4bada97b241e9b2a5278f5f5365f2/users/762e83bc36254e9c84b5f9cb929d73ac/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:11.106892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b55ba-adde-48bc-b88c-a164b32da01e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:11.107160 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b55ba-adde-48bc-b88c-a164b32da01e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:11.107438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b55ba-adde-48bc-b88c-a164b32da01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:11.159093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-df8b55ba-adde-48bc-b88c-a164b32da01e None None] MFA Rules not processed for user `762e83bc36254e9c84b5f9cb929d73ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:11.160437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-df8b55ba-adde-48bc-b88c-a164b32da01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:11.183260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:11.249502 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 839/1688] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:11.254261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fb56d-e254-464a-adfb-293a635e1bc3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:11.454643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fb56d-e254-464a-adfb-293a635e1bc3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:11.470760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e91b5-4bf2-495a-9650-dfcc8ae81dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:11.471973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e91b5-4bf2-495a-9650-dfcc8ae81dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:11.472467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e91b5-4bf2-495a-9650-dfcc8ae81dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:11.472755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e91b5-4bf2-495a-9650-dfcc8ae81dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:11.473464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e91b5-4bf2-495a-9650-dfcc8ae81dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:11.478896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e91b5-4bf2-495a-9650-dfcc8ae81dbd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:11.565970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 850/1689] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:11 2022] POST /identity/v3/projects => generated 387 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:11.567505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b55ba-adde-48bc-b88c-a164b32da01e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:11.589123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b55ba-adde-48bc-b88c-a164b32da01e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:11.704251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7df7b-1df3-46bc-91c1-9cd6a71b697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:11.705374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7df7b-1df3-46bc-91c1-9cd6a71b697a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:11.706363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7df7b-1df3-46bc-91c1-9cd6a71b697a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:11.706734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7df7b-1df3-46bc-91c1-9cd6a71b697a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:11.707543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7df7b-1df3-46bc-91c1-9cd6a71b697a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:11.709926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7df7b-1df3-46bc-91c1-9cd6a71b697a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:11.764624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 840/1690] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:11 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:11.771034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e91b5-4bf2-495a-9650-dfcc8ae81dbd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:11.806729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e91b5-4bf2-495a-9650-dfcc8ae81dbd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:11.821965 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca7554-b100-40de-a238-5c76deebbe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:11.822775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca7554-b100-40de-a238-5c76deebbe60 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:11.823163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca7554-b100-40de-a238-5c76deebbe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:11.823545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca7554-b100-40de-a238-5c76deebbe60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:11.827926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca7554-b100-40de-a238-5c76deebbe60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:11.832118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca7554-b100-40de-a238-5c76deebbe60 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:11.873387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 851/1691] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:11.884655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7df7b-1df3-46bc-91c1-9cd6a71b697a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:11.890813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372a26c-b29b-47c8-b591-b8948e974d92 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:11.891334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372a26c-b29b-47c8-b591-b8948e974d92 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:11.891814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372a26c-b29b-47c8-b591-b8948e974d92 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:11.899618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7df7b-1df3-46bc-91c1-9cd6a71b697a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:11.901774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 852/1692] 10.222.0.32 () {54 vars in 960 bytes} [Mon May 30 12:04:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:04:11.907131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fd552-2389-47f8-aa6f-2be1f035d0cf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:11.912914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a458640-5636-46ae-8c48-0b5b49553ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:11.913838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a458640-5636-46ae-8c48-0b5b49553ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:11.914263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a458640-5636-46ae-8c48-0b5b49553ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:11.914543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a458640-5636-46ae-8c48-0b5b49553ed0 admin admin] PATH_INFO: `/v3/projects/f210e206ea50473aa754762bafa6cb2f/users/ec8496b13d9e44b98bc62afb1b142ac6/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:11.914951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fd552-2389-47f8-aa6f-2be1f035d0cf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:11.916247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fd552-2389-47f8-aa6f-2be1f035d0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:11.949231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a458640-5636-46ae-8c48-0b5b49553ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f210e206ea50473aa754762bafa6cb2f, user_id=ec8496b13d9e44b98bc62afb1b142ac6, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:11.952104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a458640-5636-46ae-8c48-0b5b49553ed0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:11.959776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e70fd552-2389-47f8-aa6f-2be1f035d0cf None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:11.963288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e70fd552-2389-47f8-aa6f-2be1f035d0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:11.994749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 841/1693] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:11 2022] PUT /identity/v3/projects/f210e206ea50473aa754762bafa6cb2f/users/ec8496b13d9e44b98bc62afb1b142ac6/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:11.996073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a458640-5636-46ae-8c48-0b5b49553ed0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:12.092504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 853/1694] 10.222.0.32 () {58 vars in 1057 bytes} [Mon May 30 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:12.101420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fd552-2389-47f8-aa6f-2be1f035d0cf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:12.115231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a458640-5636-46ae-8c48-0b5b49553ed0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:12.122669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b954fd-7494-40c4-8c42-afc2f1945bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:12.123445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b954fd-7494-40c4-8c42-afc2f1945bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:12.123995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b954fd-7494-40c4-8c42-afc2f1945bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:12.125066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fd552-2389-47f8-aa6f-2be1f035d0cf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:12.127900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b954fd-7494-40c4-8c42-afc2f1945bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:12.129746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b954fd-7494-40c4-8c42-afc2f1945bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:12.131711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b954fd-7494-40c4-8c42-afc2f1945bcb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:12.145762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 842/1695] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:12.150074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b954fd-7494-40c4-8c42-afc2f1945bcb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:12.167660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b954fd-7494-40c4-8c42-afc2f1945bcb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:12.180927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f56bae-cd71-49dc-93cc-70d397476f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:12.184525 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f56bae-cd71-49dc-93cc-70d397476f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:12.189208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f56bae-cd71-49dc-93cc-70d397476f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:12.194354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f56bae-cd71-49dc-93cc-70d397476f4e admin admin] PATH_INFO: `/v3/projects/f210e206ea50473aa754762bafa6cb2f/users/ec8496b13d9e44b98bc62afb1b142ac6/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:12.200435 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f56bae-cd71-49dc-93cc-70d397476f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f210e206ea50473aa754762bafa6cb2f, user_id=ec8496b13d9e44b98bc62afb1b142ac6, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:12.207140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f56bae-cd71-49dc-93cc-70d397476f4e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:12.241644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 843/1696] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:12 2022] PUT /identity/v3/projects/f210e206ea50473aa754762bafa6cb2f/users/ec8496b13d9e44b98bc62afb1b142ac6/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:12.253069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc98c-0339-49dc-972f-22f873c43b51 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:12.253365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc98c-0339-49dc-972f-22f873c43b51 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:12.254098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc98c-0339-49dc-972f-22f873c43b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:12.283054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:12.295321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:12.298255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:12.298551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:12.302039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:12.311826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:12.316921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-11cdc98c-0339-49dc-972f-22f873c43b51 None None] MFA Rules not processed for user `ec8496b13d9e44b98bc62afb1b142ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:12.318551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-11cdc98c-0339-49dc-972f-22f873c43b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:12.343216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:12.349527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:12.399627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:12.429638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 854/1697] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:12.572852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 844/1698] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:12.597878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:12.599405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:12.777043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d9821-302e-4eaf-a704-dbdc996cd4fe service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:12.899514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bb39c-4494-4111-8e83-1ff275f40408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:12.900242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb39c-4494-4111-8e83-1ff275f40408 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:12.900535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb39c-4494-4111-8e83-1ff275f40408 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:12.900818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb39c-4494-4111-8e83-1ff275f40408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:12.912291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bb39c-4494-4111-8e83-1ff275f40408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:12.913977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bb39c-4494-4111-8e83-1ff275f40408 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:12.954494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 855/1699] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:13.114076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdc98c-0339-49dc-972f-22f873c43b51 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:13.286455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdc98c-0339-49dc-972f-22f873c43b51 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:13.311895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:13.313781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:13.315264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:13.316610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] PATH_INFO: `/v3/users/762e83bc36254e9c84b5f9cb929d73ac` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:13.320430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=762e83bc36254e9c84b5f9cb929d73ac)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:13.329092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:13.462418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762e83bc36254e9c84b5f9cb929d73ac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:13.500972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762e83bc36254e9c84b5f9cb929d73ac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:13.504674 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762e83bc36254e9c84b5f9cb929d73ac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:13.526795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762e83bc36254e9c84b5f9cb929d73ac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:13.529701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762e83bc36254e9c84b5f9cb929d73ac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:13.577207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 845/1700] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:13 2022] DELETE /identity/v3/users/762e83bc36254e9c84b5f9cb929d73ac => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:13.582278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf0a5-473f-4bc8-8c15-46e5ba2bb68a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:13.582826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf0a5-473f-4bc8-8c15-46e5ba2bb68a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:13.583083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaf0a5-473f-4bc8-8c15-46e5ba2bb68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:13.632388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-73eaf0a5-473f-4bc8-8c15-46e5ba2bb68a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:13.634108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-73eaf0a5-473f-4bc8-8c15-46e5ba2bb68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:13.793616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 856/1701] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:13.847375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:13.850171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:13.882440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:13.994775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0cef-379e-4cd9-b66c-42aadbc0efc2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:14.005477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22445-85b2-4091-b537-3987dc1f4b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:14.008438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22445-85b2-4091-b537-3987dc1f4b7f service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:14.009865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22445-85b2-4091-b537-3987dc1f4b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:14.010266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22445-85b2-4091-b537-3987dc1f4b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:14.038063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc22445-85b2-4091-b537-3987dc1f4b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:14.039618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc22445-85b2-4091-b537-3987dc1f4b7f service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:14.065986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 846/1702] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:15.500853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf0a5-473f-4bc8-8c15-46e5ba2bb68a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:15.503727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-73eaf0a5-473f-4bc8-8c15-46e5ba2bb68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:15.528118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaf0a5-473f-4bc8-8c15-46e5ba2bb68a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:15.551321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:15.552259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:15.552555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:15.552836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] PATH_INFO: `/v3/projects/c3d4bada97b241e9b2a5278f5f5365f2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:15.554179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d4bada97b241e9b2a5278f5f5365f2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:15.556215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:15.690565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d4bada97b241e9b2a5278f5f5365f2', 'request_id': 'req-e91e107f-221e-497a-8ad9-9cc3575f8eaf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:15.701165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:15.712473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d4bada97b241e9b2a5278f5f5365f2', 'request_id': 'req-e91e107f-221e-497a-8ad9-9cc3575f8eaf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:15.734261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d4bada97b241e9b2a5278f5f5365f2', 'request_id': 'req-e91e107f-221e-497a-8ad9-9cc3575f8eaf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:15.779930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] The token cache is being invalidate because project c3d4bada97b241e9b2a5278f5f5365f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:15.780322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:15.781037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e91e107f-221e-497a-8ad9-9cc3575f8eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:15.788880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 857/1703] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:15 2022] DELETE /identity/v3/projects/c3d4bada97b241e9b2a5278f5f5365f2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:15.793565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22445-85b2-4091-b537-3987dc1f4b7f service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:15.808412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22445-85b2-4091-b537-3987dc1f4b7f service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:15.974635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:15.975673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:15.975964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:15.976404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] PATH_INFO: `/v3/users/ec8496b13d9e44b98bc62afb1b142ac6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:15.979518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8496b13d9e44b98bc62afb1b142ac6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:15.981547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:16.125362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8496b13d9e44b98bc62afb1b142ac6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:16.190411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8496b13d9e44b98bc62afb1b142ac6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:16.191442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8496b13d9e44b98bc62afb1b142ac6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:16.229724 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8496b13d9e44b98bc62afb1b142ac6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:16.230703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8496b13d9e44b98bc62afb1b142ac6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:16.277652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 847/1704] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:15 2022] DELETE /identity/v3/users/ec8496b13d9e44b98bc62afb1b142ac6 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:04:16.715443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c8a6-e54f-4877-9368-a41ad374dd74 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:16.715974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c8a6-e54f-4877-9368-a41ad374dd74 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:16.716064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c8a6-e54f-4877-9368-a41ad374dd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:16.820413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c505c8a6-e54f-4877-9368-a41ad374dd74 None None] MFA Rules not processed for user `7575179edbda45208561c983fc5b956c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:16.822306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c505c8a6-e54f-4877-9368-a41ad374dd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:17.006172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 858/1705] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:17.012613 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:17.014069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:17.133309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:17.229538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6691073-84b3-4561-84c9-c8b85d6309aa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:17.244899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6e127-9c46-4334-9e4f-30549386912c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:17.245615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e127-9c46-4334-9e4f-30549386912c service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:17.245899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e127-9c46-4334-9e4f-30549386912c service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:17.246181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6e127-9c46-4334-9e4f-30549386912c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:17.278187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6e127-9c46-4334-9e4f-30549386912c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:17.280013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6e127-9c46-4334-9e4f-30549386912c service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:17.323387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 848/1706] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:17.425389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505c8a6-e54f-4877-9368-a41ad374dd74 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:17.426935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c505c8a6-e54f-4877-9368-a41ad374dd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:17.642673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505c8a6-e54f-4877-9368-a41ad374dd74 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:17.661814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:17.662757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:17.663232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:17.663533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] PATH_INFO: `/v3/projects/f210e206ea50473aa754762bafa6cb2f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:17.673373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f210e206ea50473aa754762bafa6cb2f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:17.675439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:17.850637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f210e206ea50473aa754762bafa6cb2f', 'request_id': 'req-c8c591c0-b45a-4399-98db-19de0be50328'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:17.871948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f210e206ea50473aa754762bafa6cb2f', 'request_id': 'req-c8c591c0-b45a-4399-98db-19de0be50328'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:17.891288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f210e206ea50473aa754762bafa6cb2f', 'request_id': 'req-c8c591c0-b45a-4399-98db-19de0be50328'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:17.929302 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] The token cache is being invalidate because project f210e206ea50473aa754762bafa6cb2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:17.929603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:17.930473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c8c591c0-b45a-4399-98db-19de0be50328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:17.932306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 859/1707] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:17 2022] DELETE /identity/v3/projects/f210e206ea50473aa754762bafa6cb2f => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:17.938361 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673f88-cc68-40d3-82d4-380f20e89505 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:17.938844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673f88-cc68-40d3-82d4-380f20e89505 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:17.939273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673f88-cc68-40d3-82d4-380f20e89505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:17.981009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-de673f88-cc68-40d3-82d4-380f20e89505 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:17.982499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-de673f88-cc68-40d3-82d4-380f20e89505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:18.061452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd23e-5be5-4368-8d8a-6c1d3306fb1a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:18.061752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd23e-5be5-4368-8d8a-6c1d3306fb1a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:18.062207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcd23e-5be5-4368-8d8a-6c1d3306fb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:18.069030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:18.109361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8fdcd23e-5be5-4368-8d8a-6c1d3306fb1a None None] MFA Rules not processed for user `c76e8d7bfdc04ea79c83ca7fb6b9ade1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:18.110643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8fdcd23e-5be5-4368-8d8a-6c1d3306fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:18.189381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 849/1708] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:18.196687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869afbcc-262a-45e0-a95e-4178cde91713 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:18.202396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869afbcc-262a-45e0-a95e-4178cde91713 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:18.202903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869afbcc-262a-45e0-a95e-4178cde91713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:18.270109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-869afbcc-262a-45e0-a95e-4178cde91713 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:18.271364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-869afbcc-262a-45e0-a95e-4178cde91713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:18.308534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 860/1709] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:18.317030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 850/1710] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:18.321627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcd23e-5be5-4368-8d8a-6c1d3306fb1a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:18.358666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcd23e-5be5-4368-8d8a-6c1d3306fb1a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:18.381451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:18.382161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:18.382453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:18.382738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:18.383494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:18.383815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:18.385545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:18.435479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 861/1711] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:18.439855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869afbcc-262a-45e0-a95e-4178cde91713 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:18.452681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869afbcc-262a-45e0-a95e-4178cde91713 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:18.490163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:18.490857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:18.491145 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:18.491416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:18.492321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:18.494526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:18.628146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 851/1712] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:18 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:18.632502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:18.639132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:18.648759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:18.659589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f8d48-5fc8-4893-9e2f-f521d38b3938 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:18.863737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c273-3443-45c8-a75c-ed404dacd1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:18.868382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c273-3443-45c8-a75c-ed404dacd1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:18.868899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c273-3443-45c8-a75c-ed404dacd1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:18.869330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c273-3443-45c8-a75c-ed404dacd1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:18.871318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5c273-3443-45c8-a75c-ed404dacd1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:18.880060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5c273-3443-45c8-a75c-ed404dacd1a1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:18.893361 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdd6f3-bd37-4cce-ba95-2be839484cd1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:18.915072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:18.916559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:18.917043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:18.917713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:18.929136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:18.950947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 862/1713] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:18 2022] POST /identity/v3/users => generated 288 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:18.956323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c273-3443-45c8-a75c-ed404dacd1a1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:19.018385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c273-3443-45c8-a75c-ed404dacd1a1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:19.047286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17e66-5c7b-47bf-bbad-fbbf5f34554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:19.048095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17e66-5c7b-47bf-bbad-fbbf5f34554c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:19.048415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17e66-5c7b-47bf-bbad-fbbf5f34554c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:19.048699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17e66-5c7b-47bf-bbad-fbbf5f34554c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:19.053378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17e66-5c7b-47bf-bbad-fbbf5f34554c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:19.056877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17e66-5c7b-47bf-bbad-fbbf5f34554c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:19.090628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 863/1714] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:19.094102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b97a-0113-43bc-b123-463cb8c94396 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:19.094102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b97a-0113-43bc-b123-463cb8c94396 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:19.094102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952b97a-0113-43bc-b123-463cb8c94396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:19.134016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:19.154814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b952b97a-0113-43bc-b123-463cb8c94396 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:19.156925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b952b97a-0113-43bc-b123-463cb8c94396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:19.220333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:19.222508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:19.259122 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 852/1715] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:19.260234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:19.297792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36480c-93e1-4869-8b79-1bb46b85db2d service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:19.311158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496c6b1-f9e7-436c-b9e6-edb0dfe0cf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:19.311859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496c6b1-f9e7-436c-b9e6-edb0dfe0cf83 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:19.312181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496c6b1-f9e7-436c-b9e6-edb0dfe0cf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:19.312458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496c6b1-f9e7-436c-b9e6-edb0dfe0cf83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:19.314187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496c6b1-f9e7-436c-b9e6-edb0dfe0cf83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:19.320305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496c6b1-f9e7-436c-b9e6-edb0dfe0cf83 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:19.352651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 853/1716] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:19.359402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 864/1717] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:19.360465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496c6b1-f9e7-436c-b9e6-edb0dfe0cf83 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:19.385169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496c6b1-f9e7-436c-b9e6-edb0dfe0cf83 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:19.404127 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:19.405070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:19.405510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:19.405921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] PATH_INFO: `/v3/projects/529bcc9cae8841cf9d9948fd713defda/users/6d8b74cde0bd4e5d9dca41253f26af48/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:19.444483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] RBAC: Authorizing `identity:create_grant(project_id=529bcc9cae8841cf9d9948fd713defda, user_id=6d8b74cde0bd4e5d9dca41253f26af48, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:19.447242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:19.496780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 854/1718] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:19 2022] PUT /identity/v3/projects/529bcc9cae8841cf9d9948fd713defda/users/6d8b74cde0bd4e5d9dca41253f26af48/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:19.498045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdc314-56c1-469a-ab4b-3a45ebb4e674 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:19.498347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdc314-56c1-469a-ab4b-3a45ebb4e674 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:19.498658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdc314-56c1-469a-ab4b-3a45ebb4e674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:19.563204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7cfdc314-56c1-469a-ab4b-3a45ebb4e674 None None] MFA Rules not processed for user `6d8b74cde0bd4e5d9dca41253f26af48`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:19.565774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7cfdc314-56c1-469a-ab4b-3a45ebb4e674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:19.593020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:19.694589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:19.696462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:19.754953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 865/1719] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:19.768511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdc314-56c1-469a-ab4b-3a45ebb4e674 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:19.770522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7cfdc314-56c1-469a-ab4b-3a45ebb4e674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:19.901183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a63f-fee0-4062-bfae-e17ce144085b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:19.908684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3338d1b-3022-4b57-af10-85de1a532807 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:19.909649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338d1b-3022-4b57-af10-85de1a532807 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:19.910115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338d1b-3022-4b57-af10-85de1a532807 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:19.910542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338d1b-3022-4b57-af10-85de1a532807 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:19.929032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdc314-56c1-469a-ab4b-3a45ebb4e674 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:19.940577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa702bc-a09d-4147-9233-abc76d563f93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad37bdce39f64836b0cd14613cabe91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:19.941648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa702bc-a09d-4147-9233-abc76d563f93 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:19.941938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa702bc-a09d-4147-9233-abc76d563f93 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:19.942398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa702bc-a09d-4147-9233-abc76d563f93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:19.965440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa702bc-a09d-4147-9233-abc76d563f93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:19.967314 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa702bc-a09d-4147-9233-abc76d563f93 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:20.000907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 866/1720] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:20.103039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3338d1b-3022-4b57-af10-85de1a532807 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:20.104948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3338d1b-3022-4b57-af10-85de1a532807 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:20.123011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 855/1721] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:04:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 30 12:04:20.383545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49684752-224b-43de-88c1-248072bb6870 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:20.383837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49684752-224b-43de-88c1-248072bb6870 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:20.384139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49684752-224b-43de-88c1-248072bb6870 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:20.386145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 867/1722] 10.222.0.32 () {54 vars in 912 bytes} [Mon May 30 12:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:04:20.403248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcbd32-94c5-4a2a-810d-3f0be3272fcc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:20.403767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcbd32-94c5-4a2a-810d-3f0be3272fcc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:20.403767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcbd32-94c5-4a2a-810d-3f0be3272fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:20.463006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a7dcbd32-94c5-4a2a-810d-3f0be3272fcc None None] MFA Rules not processed for user `15ea38389eee4472a8917023643be9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:20.466415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a7dcbd32-94c5-4a2a-810d-3f0be3272fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:20.689000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:20.731841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 856/1723] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:04:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 30 12:04:20.897195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49684752-224b-43de-88c1-248072bb6870 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:20.899160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-49684752-224b-43de-88c1-248072bb6870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:21.063815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcbd32-94c5-4a2a-810d-3f0be3272fcc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:21.066061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a7dcbd32-94c5-4a2a-810d-3f0be3272fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:21.073736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49684752-224b-43de-88c1-248072bb6870 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:21.080452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcbd32-94c5-4a2a-810d-3f0be3272fcc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:21.098541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:21.101151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:21.101446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:21.101735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] PATH_INFO: `/v3/users/829c8ebd4eaa4b47b130ce6c2a8f80e5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:21.114371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] RBAC: Authorizing `identity:delete_user(user_id=829c8ebd4eaa4b47b130ce6c2a8f80e5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:21.118938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:21.119979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d49d6-f1dd-4060-b4f2-8a132eebffa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:21.121593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d49d6-f1dd-4060-b4f2-8a132eebffa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:21.122764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d49d6-f1dd-4060-b4f2-8a132eebffa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:21.124015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d49d6-f1dd-4060-b4f2-8a132eebffa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:21.175053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d49d6-f1dd-4060-b4f2-8a132eebffa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:21.191793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d49d6-f1dd-4060-b4f2-8a132eebffa0 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:21.223418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 857/1724] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:21.232660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '829c8ebd4eaa4b47b130ce6c2a8f80e5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:21.249782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:21.276972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '829c8ebd4eaa4b47b130ce6c2a8f80e5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:21.280434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '829c8ebd4eaa4b47b130ce6c2a8f80e5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:21.282201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '829c8ebd4eaa4b47b130ce6c2a8f80e5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:21.315757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '829c8ebd4eaa4b47b130ce6c2a8f80e5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:21.365022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 868/1725] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:20 2022] DELETE /identity/v3/users/829c8ebd4eaa4b47b130ce6c2a8f80e5 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:21.370774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb4664-f579-4204-af14-7fecf4acab0f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:21.371340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb4664-f579-4204-af14-7fecf4acab0f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:21.372004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb4664-f579-4204-af14-7fecf4acab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:21.416679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-96eb4664-f579-4204-af14-7fecf4acab0f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:21.421100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-96eb4664-f579-4204-af14-7fecf4acab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:21.766736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 858/1726] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:21.823040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:21.825020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:21.957728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77222ce-c030-49b6-813b-1192ea42247e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:21.972064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:21.973074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:21.973521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:21.973955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:22.014937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:22.016509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:22.056939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 869/1727] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:23.389700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16305-c2d2-44d0-822d-68170b1e7b6f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:23.390269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16305-c2d2-44d0-822d-68170b1e7b6f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:23.390269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16305-c2d2-44d0-822d-68170b1e7b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:23.448430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-69c16305-c2d2-44d0-822d-68170b1e7b6f None None] MFA Rules not processed for user `15ea38389eee4472a8917023643be9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:23.449714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-69c16305-c2d2-44d0-822d-68170b1e7b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:23.485997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 859/1728] 10.222.0.65 () {58 vars in 1015 bytes} [Mon May 30 12:04:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:23.699688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:23.701171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:23.703602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168e31c-a677-4b50-b2b8-25d16f5f1c86 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:23.703892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168e31c-a677-4b50-b2b8-25d16f5f1c86 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:23.704187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168e31c-a677-4b50-b2b8-25d16f5f1c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:23.726802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdc211-dd89-43a5-9526-41567f819abf service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:23.750280 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:23.751385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:23.751838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:23.752339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:23.753538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b168e31c-a677-4b50-b2b8-25d16f5f1c86 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:23.760200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b168e31c-a677-4b50-b2b8-25d16f5f1c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:23.776312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:23.788310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:23.814346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 870/1729] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:24.060452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 860/1730] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:04:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:24.076099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:24.077863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:24.335935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17591a-c9fe-4864-b8d5-be7f3879d71e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:24.350408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:24.351097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:24.351392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:24.351674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:24.381160 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e5256-71e0-40c3-bfe4-1f42f09237bb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:24.381803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e5256-71e0-40c3-bfe4-1f42f09237bb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:24.382365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e5256-71e0-40c3-bfe4-1f42f09237bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:24.387370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:24.396026 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:24.421982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 871/1731] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:04:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:04:24.488379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-de1e5256-71e0-40c3-bfe4-1f42f09237bb None None] MFA Rules not processed for user `bc47c5bf77894aefa6ecde115ed9f88f`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:24.489695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-de1e5256-71e0-40c3-bfe4-1f42f09237bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:24.797017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 861/1732] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:24.805947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:24.808224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:24.989829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e5256-71e0-40c3-bfe4-1f42f09237bb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:24.991327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-de1e5256-71e0-40c3-bfe4-1f42f09237bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:25.039550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e5256-71e0-40c3-bfe4-1f42f09237bb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:25.068954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:25.069916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:25.070217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:25.070499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] PATH_INFO: `/v3/projects/96630b03d2094e1494969734ef6cd2a6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:25.083441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96630b03d2094e1494969734ef6cd2a6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:25.098005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:25.109805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:25.147957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:25.223915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e941c44-a164-40c6-8df3-50c36cdc24cd service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:25.244359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eeaba-91be-4db1-8ffa-4da591fb54ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:25.247385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eeaba-91be-4db1-8ffa-4da591fb54ae service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:25.247864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eeaba-91be-4db1-8ffa-4da591fb54ae service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:25.248335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eeaba-91be-4db1-8ffa-4da591fb54ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:25.272763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67eeaba-91be-4db1-8ffa-4da591fb54ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:25.277144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67eeaba-91be-4db1-8ffa-4da591fb54ae service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:25.332988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 872/1733] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:25.385627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96630b03d2094e1494969734ef6cd2a6', 'request_id': 'req-527e5138-4791-4491-ba4f-76bc74f4c4b0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:25.409703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96630b03d2094e1494969734ef6cd2a6', 'request_id': 'req-527e5138-4791-4491-ba4f-76bc74f4c4b0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:25.425079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96630b03d2094e1494969734ef6cd2a6', 'request_id': 'req-527e5138-4791-4491-ba4f-76bc74f4c4b0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:25.451380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] The token cache is being invalidate because project 96630b03d2094e1494969734ef6cd2a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:25.452753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:25.459231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-527e5138-4791-4491-ba4f-76bc74f4c4b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:25.472969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 862/1734] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:24 2022] DELETE /identity/v3/projects/96630b03d2094e1494969734ef6cd2a6 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:25.474160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eeaba-91be-4db1-8ffa-4da591fb54ae service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:25.505711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abde13-c5ef-455a-b45e-d4cd58d7f609 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:25.506239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abde13-c5ef-455a-b45e-d4cd58d7f609 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:25.506239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abde13-c5ef-455a-b45e-d4cd58d7f609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:25.523338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eeaba-91be-4db1-8ffa-4da591fb54ae service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:25.554788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-73abde13-c5ef-455a-b45e-d4cd58d7f609 None None] MFA Rules not processed for user `7575179edbda45208561c983fc5b956c`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:25.559343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-73abde13-c5ef-455a-b45e-d4cd58d7f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:25.777149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:25.778804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:25.779410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:25.780027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] PATH_INFO: `/v3/users/a7ced8e0f2cb4582ba4daccabfd6670b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:25.784389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ced8e0f2cb4582ba4daccabfd6670b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:25.789117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:25.902387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 863/1735] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:25.943750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ced8e0f2cb4582ba4daccabfd6670b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:25.987415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ced8e0f2cb4582ba4daccabfd6670b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:25.993190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ced8e0f2cb4582ba4daccabfd6670b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:25.993931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ced8e0f2cb4582ba4daccabfd6670b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:26.009037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1d404c66-93dc-4c11-a4eb-bd1461181adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ced8e0f2cb4582ba4daccabfd6670b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:26.046750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 873/1736] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:25 2022] DELETE /identity/v3/users/a7ced8e0f2cb4582ba4daccabfd6670b => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:04:26.254746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afc09-e419-46f9-84e3-40564b99ca2e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:26.255102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afc09-e419-46f9-84e3-40564b99ca2e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:26.255458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afc09-e419-46f9-84e3-40564b99ca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:26.384925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-467afc09-e419-46f9-84e3-40564b99ca2e None None] MFA Rules not processed for user `166b42a5d6c04659afb20c088dfead76`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:26.386583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-467afc09-e419-46f9-84e3-40564b99ca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:26.718939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 864/1737] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:26.729899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a153d-b71d-4232-b226-e7a7a01dfde5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:26.730256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a153d-b71d-4232-b226-e7a7a01dfde5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:26.730479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a153d-b71d-4232-b226-e7a7a01dfde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:26.773395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-af5a153d-b71d-4232-b226-e7a7a01dfde5 None None] MFA Rules not processed for user `166b42a5d6c04659afb20c088dfead76`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:26.774927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af5a153d-b71d-4232-b226-e7a7a01dfde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:26.794485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 874/1738] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:27.150680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467afc09-e419-46f9-84e3-40564b99ca2e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:27.152510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-467afc09-e419-46f9-84e3-40564b99ca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:27.262540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:27.423220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467afc09-e419-46f9-84e3-40564b99ca2e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:27.444099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7364f-9b4b-4c82-9a21-4604675cf5c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:27.444888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7364f-9b4b-4c82-9a21-4604675cf5c8 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:27.445328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7364f-9b4b-4c82-9a21-4604675cf5c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:27.445715 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7364f-9b4b-4c82-9a21-4604675cf5c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:27.458958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7364f-9b4b-4c82-9a21-4604675cf5c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:27.460844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7364f-9b4b-4c82-9a21-4604675cf5c8 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:27.509732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 865/1739] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:27.879209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a153d-b71d-4232-b226-e7a7a01dfde5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:27.881623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af5a153d-b71d-4232-b226-e7a7a01dfde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:28.015020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:28.018312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a33dd-188e-49a8-aa03-5236c07e3a4f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.022586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a33dd-188e-49a8-aa03-5236c07e3a4f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.027461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a33dd-188e-49a8-aa03-5236c07e3a4f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.035066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 866/1740] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 12:04:28 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:04:28.066417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a153d-b71d-4232-b226-e7a7a01dfde5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:28.069105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341714b-b42c-4999-8c69-2b8ba4c247bb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.069970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341714b-b42c-4999-8c69-2b8ba4c247bb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.070280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341714b-b42c-4999-8c69-2b8ba4c247bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.088290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:28.089379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.089769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.090056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] PATH_INFO: `/v3/projects/de1af30de2034b34bbf58ccbb5ffd845` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.091683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1af30de2034b34bbf58ccbb5ffd845)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:28.093746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:28.112037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a341714b-b42c-4999-8c69-2b8ba4c247bb None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:28.114309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a341714b-b42c-4999-8c69-2b8ba4c247bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:28.142197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 867/1741] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 12:04:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:28.200864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1af30de2034b34bbf58ccbb5ffd845', 'request_id': 'req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:28.216219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1af30de2034b34bbf58ccbb5ffd845', 'request_id': 'req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:28.225679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1af30de2034b34bbf58ccbb5ffd845', 'request_id': 'req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:28.240363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] The token cache is being invalidate because project de1af30de2034b34bbf58ccbb5ffd845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:28.240664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:28.241357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95390fa1-d2a5-4d5e-9052-1db37dee8fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:28.246346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 875/1742] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:27 2022] DELETE /identity/v3/projects/de1af30de2034b34bbf58ccbb5ffd845 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:28.248293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fa947-1cb1-44da-81b8-d694b11c64d2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.248585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fa947-1cb1-44da-81b8-d694b11c64d2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.248865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fa947-1cb1-44da-81b8-d694b11c64d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.280605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e85fa947-1cb1-44da-81b8-d694b11c64d2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:28.284113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e85fa947-1cb1-44da-81b8-d694b11c64d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:28.369266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 868/1743] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:28.379138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b610f-a7e7-486a-805a-d2bb2d0cda9f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.379650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b610f-a7e7-486a-805a-d2bb2d0cda9f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.380124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b610f-a7e7-486a-805a-d2bb2d0cda9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.435225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f17b610f-a7e7-486a-805a-d2bb2d0cda9f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:28.437198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f17b610f-a7e7-486a-805a-d2bb2d0cda9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:28.450003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 876/1744] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:28.458438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fa947-1cb1-44da-81b8-d694b11c64d2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:28.474254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fa947-1cb1-44da-81b8-d694b11c64d2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:28.481571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:28.482396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.482784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.484598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.485989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:28.487163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:28.490247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:28.509785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 869/1745] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:28.513156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b610f-a7e7-486a-805a-d2bb2d0cda9f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:28.528507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b610f-a7e7-486a-805a-d2bb2d0cda9f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:28.537967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00beb14-b1d1-4f27-933b-879ee1149810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:28.538759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00beb14-b1d1-4f27-933b-879ee1149810 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.539115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00beb14-b1d1-4f27-933b-879ee1149810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.539699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00beb14-b1d1-4f27-933b-879ee1149810 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.542003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00beb14-b1d1-4f27-933b-879ee1149810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:28.544829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00beb14-b1d1-4f27-933b-879ee1149810 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:28.602424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 877/1746] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:28 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:04:28.607082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:28.618538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506602f-2d5e-4237-9203-418694fade17 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:28.790020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7935bd-854f-4a56-87f1-25f47a4b35db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:28.790718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7935bd-854f-4a56-87f1-25f47a4b35db admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.790978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7935bd-854f-4a56-87f1-25f47a4b35db admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.791263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7935bd-854f-4a56-87f1-25f47a4b35db admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.791980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7935bd-854f-4a56-87f1-25f47a4b35db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:28.794035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7935bd-854f-4a56-87f1-25f47a4b35db admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:28.822903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 870/1747] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:28 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:04:28.826862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00beb14-b1d1-4f27-933b-879ee1149810 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:28.849110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00beb14-b1d1-4f27-933b-879ee1149810 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:28.858821 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18e193-19b1-4d4e-bd0d-b63e4375394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:28.859819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18e193-19b1-4d4e-bd0d-b63e4375394b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.860301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18e193-19b1-4d4e-bd0d-b63e4375394b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.860749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18e193-19b1-4d4e-bd0d-b63e4375394b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.863729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18e193-19b1-4d4e-bd0d-b63e4375394b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:28.866701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18e193-19b1-4d4e-bd0d-b63e4375394b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:28.888813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 878/1748] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:28.889856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7935bd-854f-4a56-87f1-25f47a4b35db admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:28.901019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7935bd-854f-4a56-87f1-25f47a4b35db admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:28.907840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b54688-6e7d-4083-855e-e9ed6a7c99b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:28.909028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54688-6e7d-4083-855e-e9ed6a7c99b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.909566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54688-6e7d-4083-855e-e9ed6a7c99b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.909683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54688-6e7d-4083-855e-e9ed6a7c99b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:28.912977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b54688-6e7d-4083-855e-e9ed6a7c99b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:28.912977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b54688-6e7d-4083-855e-e9ed6a7c99b4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:28.944045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 871/1749] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:28.950786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18e193-19b1-4d4e-bd0d-b63e4375394b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:28.966649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18e193-19b1-4d4e-bd0d-b63e4375394b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:28.974591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf527cf-7bb8-486b-b97f-32bb77caf659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:28.987817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf527cf-7bb8-486b-b97f-32bb77caf659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:28.990721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf527cf-7bb8-486b-b97f-32bb77caf659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:28.991443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf527cf-7bb8-486b-b97f-32bb77caf659 admin admin] PATH_INFO: `/v3/projects/2c7e287254ea414b84ee49bb076c647a/users/1c22f7e7ff6a442a942d5f6c4c84c30b/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:29.017853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf527cf-7bb8-486b-b97f-32bb77caf659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7e287254ea414b84ee49bb076c647a, user_id=1c22f7e7ff6a442a942d5f6c4c84c30b, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:29.029788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf527cf-7bb8-486b-b97f-32bb77caf659 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:29.059730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 879/1750] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:28 2022] PUT /identity/v3/projects/2c7e287254ea414b84ee49bb076c647a/users/1c22f7e7ff6a442a942d5f6c4c84c30b/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:29.072240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b691ae9-e36e-4aa9-88f5-7c72a532c85d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:29.073194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b691ae9-e36e-4aa9-88f5-7c72a532c85d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:29.073479 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b691ae9-e36e-4aa9-88f5-7c72a532c85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:29.101311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:29.107896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6b691ae9-e36e-4aa9-88f5-7c72a532c85d None None] MFA Rules not processed for user `1c22f7e7ff6a442a942d5f6c4c84c30b`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:29.109564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6b691ae9-e36e-4aa9-88f5-7c72a532c85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:29.362588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 872/1751] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:29.363875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf527cf-7bb8-486b-b97f-32bb77caf659 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:29.571510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf527cf-7bb8-486b-b97f-32bb77caf659 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:29.598353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26801-766a-48ac-aacd-b1ca5c98fa85 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:29.598648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26801-766a-48ac-aacd-b1ca5c98fa85 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:29.598931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c26801-766a-48ac-aacd-b1ca5c98fa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:29.601592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383ad7f-59b1-4a47-96fa-4b359a40d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:29.602339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ad7f-59b1-4a47-96fa-4b359a40d6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:29.602640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ad7f-59b1-4a47-96fa-4b359a40d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:29.602961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ad7f-59b1-4a47-96fa-4b359a40d6d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:29.603708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383ad7f-59b1-4a47-96fa-4b359a40d6d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:29.609958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383ad7f-59b1-4a47-96fa-4b359a40d6d6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:29.632848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-00c26801-766a-48ac-aacd-b1ca5c98fa85 None None] MFA Rules not processed for user `bc47c5bf77894aefa6ecde115ed9f88f`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:29.634010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-00c26801-766a-48ac-aacd-b1ca5c98fa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:29.675965 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:29.691398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 880/1752] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:29 2022] POST /identity/v3/projects => generated 373 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:29.702598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383ad7f-59b1-4a47-96fa-4b359a40d6d6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:29.820371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 873/1753] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:29.845195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383ad7f-59b1-4a47-96fa-4b359a40d6d6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:29.851999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec007dc6-18d1-4639-bac5-92df7decf56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:29.853003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec007dc6-18d1-4639-bac5-92df7decf56f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:29.853240 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec007dc6-18d1-4639-bac5-92df7decf56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:29.853516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec007dc6-18d1-4639-bac5-92df7decf56f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:29.854210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec007dc6-18d1-4639-bac5-92df7decf56f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:29.856206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec007dc6-18d1-4639-bac5-92df7decf56f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:29.882158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 881/1754] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:29 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:29.886998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c26801-766a-48ac-aacd-b1ca5c98fa85 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:29.901142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c26801-766a-48ac-aacd-b1ca5c98fa85 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:29.907370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4e038-73b4-4a15-b20d-1c9bf141853c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:29.908695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4e038-73b4-4a15-b20d-1c9bf141853c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:29.908998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4e038-73b4-4a15-b20d-1c9bf141853c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:29.909535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4e038-73b4-4a15-b20d-1c9bf141853c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:29.910324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4e038-73b4-4a15-b20d-1c9bf141853c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:29.912185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4e038-73b4-4a15-b20d-1c9bf141853c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:29.930293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 874/1755] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:29.930555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec007dc6-18d1-4639-bac5-92df7decf56f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:29.949616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec007dc6-18d1-4639-bac5-92df7decf56f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:29.961184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc34552-3b99-4732-bb57-daedcf12530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:29.962564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc34552-3b99-4732-bb57-daedcf12530a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:29.963044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc34552-3b99-4732-bb57-daedcf12530a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:29.963334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc34552-3b99-4732-bb57-daedcf12530a admin admin] PATH_INFO: `/v3/projects/4ed1fb7d59654f5bb39885c5fb14ec15/users/953c86444d6c4aa18b6f821c2308a6ea/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:29.981493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc34552-3b99-4732-bb57-daedcf12530a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed1fb7d59654f5bb39885c5fb14ec15, user_id=953c86444d6c4aa18b6f821c2308a6ea, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:29.984437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc34552-3b99-4732-bb57-daedcf12530a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:30.003447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 882/1756] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:29 2022] PUT /identity/v3/projects/4ed1fb7d59654f5bb39885c5fb14ec15/users/953c86444d6c4aa18b6f821c2308a6ea/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:30.004763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4e038-73b4-4a15-b20d-1c9bf141853c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:30.016984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4e038-73b4-4a15-b20d-1c9bf141853c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:30.116322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42736a91-85b4-48da-bd65-b5d4638353f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:30.118050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42736a91-85b4-48da-bd65-b5d4638353f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:30.118479 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42736a91-85b4-48da-bd65-b5d4638353f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:30.119558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42736a91-85b4-48da-bd65-b5d4638353f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:30.120524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42736a91-85b4-48da-bd65-b5d4638353f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:30.122312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42736a91-85b4-48da-bd65-b5d4638353f4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:30.138314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 875/1757] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:30.143301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc34552-3b99-4732-bb57-daedcf12530a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:30.158477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc34552-3b99-4732-bb57-daedcf12530a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:30.163853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:30.165221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:30.165742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:30.166112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] PATH_INFO: `/v3/projects/4ed1fb7d59654f5bb39885c5fb14ec15/users/953c86444d6c4aa18b6f821c2308a6ea/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:30.170931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed1fb7d59654f5bb39885c5fb14ec15, user_id=953c86444d6c4aa18b6f821c2308a6ea, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:30.175991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:30.221855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 883/1758] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:30 2022] PUT /identity/v3/projects/4ed1fb7d59654f5bb39885c5fb14ec15/users/953c86444d6c4aa18b6f821c2308a6ea/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:30.223141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881d013-4bb6-450b-ab8e-df88da094592 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:30.224423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881d013-4bb6-450b-ab8e-df88da094592 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:30.225843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881d013-4bb6-450b-ab8e-df88da094592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:30.322236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5881d013-4bb6-450b-ab8e-df88da094592 None None] MFA Rules not processed for user `953c86444d6c4aa18b6f821c2308a6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:30.326338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5881d013-4bb6-450b-ab8e-df88da094592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:30.372222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:30.459251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 876/1759] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:30.466888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:30.468660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:30.586026 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca19fb5-0132-4c6d-aae4-92e32fa8845f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:30.595960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fe7d8-c978-445e-b4be-21fd8abf5a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:30.597187 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fe7d8-c978-445e-b4be-21fd8abf5a98 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:30.597805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fe7d8-c978-445e-b4be-21fd8abf5a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:30.598168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fe7d8-c978-445e-b4be-21fd8abf5a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:30.610223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fe7d8-c978-445e-b4be-21fd8abf5a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:30.612823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fe7d8-c978-445e-b4be-21fd8abf5a98 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:30.623810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 884/1760] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:31.276068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881d013-4bb6-450b-ab8e-df88da094592 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:31.451864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881d013-4bb6-450b-ab8e-df88da094592 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:31.463349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:31.464574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:31.464992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:31.465463 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] PATH_INFO: `/v3/users/1c22f7e7ff6a442a942d5f6c4c84c30b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:31.466990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c22f7e7ff6a442a942d5f6c4c84c30b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:31.473163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:31.553809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c22f7e7ff6a442a942d5f6c4c84c30b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:31.584773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c22f7e7ff6a442a942d5f6c4c84c30b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:31.586315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c22f7e7ff6a442a942d5f6c4c84c30b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:31.636990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c22f7e7ff6a442a942d5f6c4c84c30b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:31.638300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c22f7e7ff6a442a942d5f6c4c84c30b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:31.700742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 877/1761] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:31 2022] DELETE /identity/v3/users/1c22f7e7ff6a442a942d5f6c4c84c30b => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:31.706544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285612f5-461e-4809-b2e2-057062f95b4a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:31.707514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285612f5-461e-4809-b2e2-057062f95b4a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:31.707995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285612f5-461e-4809-b2e2-057062f95b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:31.758036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-285612f5-461e-4809-b2e2-057062f95b4a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:31.760694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-285612f5-461e-4809-b2e2-057062f95b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:31.909488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:31.934170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 885/1762] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:31.943647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:31.945524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:32.082898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:32.107772 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a4a30-13c7-4536-9835-6412df67536d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:32.113235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df4d2-e6b9-4fca-b50b-cea0c98d9f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:32.118923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df4d2-e6b9-4fca-b50b-cea0c98d9f23 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:32.119356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df4d2-e6b9-4fca-b50b-cea0c98d9f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:32.119752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df4d2-e6b9-4fca-b50b-cea0c98d9f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:32.139014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df4d2-e6b9-4fca-b50b-cea0c98d9f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:32.141365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df4d2-e6b9-4fca-b50b-cea0c98d9f23 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:32.175690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 878/1763] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:32.713866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285612f5-461e-4809-b2e2-057062f95b4a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:32.715456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-285612f5-461e-4809-b2e2-057062f95b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:32.748619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285612f5-461e-4809-b2e2-057062f95b4a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:32.757233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:32.758265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:32.758759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:32.759049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] PATH_INFO: `/v3/projects/2c7e287254ea414b84ee49bb076c647a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:32.764556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7e287254ea414b84ee49bb076c647a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:32.772123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:32.866942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7e287254ea414b84ee49bb076c647a', 'request_id': 'req-91883973-482b-4ec9-97a1-b67f788e66e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:32.874957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7e287254ea414b84ee49bb076c647a', 'request_id': 'req-91883973-482b-4ec9-97a1-b67f788e66e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:32.882650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7e287254ea414b84ee49bb076c647a', 'request_id': 'req-91883973-482b-4ec9-97a1-b67f788e66e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:32.897149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] The token cache is being invalidate because project 2c7e287254ea414b84ee49bb076c647a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:32.897441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:32.898104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:32.899669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 886/1764] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:32 2022] DELETE /identity/v3/projects/2c7e287254ea414b84ee49bb076c647a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:32.914349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df4d2-e6b9-4fca-b50b-cea0c98d9f23 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:32.926438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df4d2-e6b9-4fca-b50b-cea0c98d9f23 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:33.109326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:33.110246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:33.110571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:33.110997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] PATH_INFO: `/v3/users/953c86444d6c4aa18b6f821c2308a6ea` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:33.114600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953c86444d6c4aa18b6f821c2308a6ea)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:33.116623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:33.244054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953c86444d6c4aa18b6f821c2308a6ea'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.275611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953c86444d6c4aa18b6f821c2308a6ea'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.276474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953c86444d6c4aa18b6f821c2308a6ea'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.314840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953c86444d6c4aa18b6f821c2308a6ea'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.315836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-afe70520-c06b-49c9-a3ef-47ab6b009262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953c86444d6c4aa18b6f821c2308a6ea'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.363430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 879/1765] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:32 2022] DELETE /identity/v3/users/953c86444d6c4aa18b6f821c2308a6ea => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:33.797473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:33.799882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:33.813735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91883973-482b-4ec9-97a1-b67f788e66e0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:33.888856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:33.889761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:33.890050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:33.890331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] PATH_INFO: `/v3/projects/4ed1fb7d59654f5bb39885c5fb14ec15` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:33.891561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed1fb7d59654f5bb39885c5fb14ec15)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:33.893652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:33.941112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed1fb7d59654f5bb39885c5fb14ec15', 'request_id': 'req-c467e123-c14e-4dd1-b2ef-456cd9e4f835'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.948758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed1fb7d59654f5bb39885c5fb14ec15', 'request_id': 'req-c467e123-c14e-4dd1-b2ef-456cd9e4f835'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.955297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed1fb7d59654f5bb39885c5fb14ec15', 'request_id': 'req-c467e123-c14e-4dd1-b2ef-456cd9e4f835'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.965797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] The token cache is being invalidate because project 4ed1fb7d59654f5bb39885c5fb14ec15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:33.966123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.966857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c467e123-c14e-4dd1-b2ef-456cd9e4f835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:33.968788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 887/1766] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:33 2022] DELETE /identity/v3/projects/4ed1fb7d59654f5bb39885c5fb14ec15 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:33.974742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cd7af-197c-450c-90a0-009aca6be034 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:33.975338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cd7af-197c-450c-90a0-009aca6be034 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:33.975338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cd7af-197c-450c-90a0-009aca6be034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:33.996413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c11cd7af-197c-450c-90a0-009aca6be034 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:33.997706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c11cd7af-197c-450c-90a0-009aca6be034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:34.060412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:34.097797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 880/1767] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:34.100926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb054a-4b8f-431d-9c21-92734983e89e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.101577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb054a-4b8f-431d-9c21-92734983e89e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.102129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb054a-4b8f-431d-9c21-92734983e89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.122900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:34.129864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5dbb054a-4b8f-431d-9c21-92734983e89e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:34.132787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5dbb054a-4b8f-431d-9c21-92734983e89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:34.145176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 888/1768] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:34.149702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cd7af-197c-450c-90a0-009aca6be034 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.160745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cd7af-197c-450c-90a0-009aca6be034 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.165570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.166241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.166531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.166817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.167563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:34.167882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.169541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.182253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 881/1769] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:34.186691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb054a-4b8f-431d-9c21-92734983e89e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.196667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb054a-4b8f-431d-9c21-92734983e89e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.201532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193162f-51ae-48a8-a820-9a697b03a752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.202125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193162f-51ae-48a8-a820-9a697b03a752 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.202624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193162f-51ae-48a8-a820-9a697b03a752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.202914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193162f-51ae-48a8-a820-9a697b03a752 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.203614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193162f-51ae-48a8-a820-9a697b03a752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.206090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193162f-51ae-48a8-a820-9a697b03a752 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.234458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 889/1770] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:34 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:34.239421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.250485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3413868-d17b-48af-a3cd-d6860b7a1603 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.318818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc967-aa8c-46f9-afe7-706e87657a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.320082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc967-aa8c-46f9-afe7-706e87657a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.320421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc967-aa8c-46f9-afe7-706e87657a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.320699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc967-aa8c-46f9-afe7-706e87657a9e admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.321393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafc967-aa8c-46f9-afe7-706e87657a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.323381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafc967-aa8c-46f9-afe7-706e87657a9e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.346236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 882/1771] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:34 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:34.350318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193162f-51ae-48a8-a820-9a697b03a752 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.362822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193162f-51ae-48a8-a820-9a697b03a752 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.368504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ef5cf-f964-47de-b106-e7ed3c3ed9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.369283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef5cf-f964-47de-b106-e7ed3c3ed9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.369773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef5cf-f964-47de-b106-e7ed3c3ed9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.370209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef5cf-f964-47de-b106-e7ed3c3ed9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.371022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ef5cf-f964-47de-b106-e7ed3c3ed9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.372787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ef5cf-f964-47de-b106-e7ed3c3ed9d7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.385837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 890/1772] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:34.390204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc967-aa8c-46f9-afe7-706e87657a9e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.401349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc967-aa8c-46f9-afe7-706e87657a9e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.406069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bd16b-4e8f-4345-914a-0f447799aad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.406750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd16b-4e8f-4345-914a-0f447799aad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.407040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd16b-4e8f-4345-914a-0f447799aad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.407321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bd16b-4e8f-4345-914a-0f447799aad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.409048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bd16b-4e8f-4345-914a-0f447799aad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.410863 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bd16b-4e8f-4345-914a-0f447799aad2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.423219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 883/1773] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:34.430920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ef5cf-f964-47de-b106-e7ed3c3ed9d7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.450750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ef5cf-f964-47de-b106-e7ed3c3ed9d7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.474320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b6173-089e-4cfe-a165-bd4421a7b5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.475253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b6173-089e-4cfe-a165-bd4421a7b5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.475540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b6173-089e-4cfe-a165-bd4421a7b5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.475847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b6173-089e-4cfe-a165-bd4421a7b5fa admin admin] PATH_INFO: `/v3/projects/5871b927396d4feead42cd97beaa7ec4/users/6567d66806aa4ee1b76e9723fd24c1fa/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.495474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b6173-089e-4cfe-a165-bd4421a7b5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5871b927396d4feead42cd97beaa7ec4, user_id=6567d66806aa4ee1b76e9723fd24c1fa, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.498392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b6173-089e-4cfe-a165-bd4421a7b5fa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.521441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 891/1774] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:34 2022] PUT /identity/v3/projects/5871b927396d4feead42cd97beaa7ec4/users/6567d66806aa4ee1b76e9723fd24c1fa/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:34.531750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a2f53-fde8-44aa-a125-8d62d06ed355 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.532070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a2f53-fde8-44aa-a125-8d62d06ed355 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.532385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a2f53-fde8-44aa-a125-8d62d06ed355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.560299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-638a2f53-fde8-44aa-a125-8d62d06ed355 None None] MFA Rules not processed for user `6567d66806aa4ee1b76e9723fd24c1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:34.561562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-638a2f53-fde8-44aa-a125-8d62d06ed355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:34.649430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 884/1775] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:34.654434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b6173-089e-4cfe-a165-bd4421a7b5fa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.735027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b6173-089e-4cfe-a165-bd4421a7b5fa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.740760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eef48-889d-4a50-b5e6-cfdef635037f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.741437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eef48-889d-4a50-b5e6-cfdef635037f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.741727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eef48-889d-4a50-b5e6-cfdef635037f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.742015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eef48-889d-4a50-b5e6-cfdef635037f admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.742718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eef48-889d-4a50-b5e6-cfdef635037f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.744726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eef48-889d-4a50-b5e6-cfdef635037f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.785913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 892/1776] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:34 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:34.790523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a2f53-fde8-44aa-a125-8d62d06ed355 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.802624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a2f53-fde8-44aa-a125-8d62d06ed355 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.868951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:34.885543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db82ec-ca16-4d36-a347-aba02ac5734f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.887304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db82ec-ca16-4d36-a347-aba02ac5734f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.887607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db82ec-ca16-4d36-a347-aba02ac5734f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.888181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db82ec-ca16-4d36-a347-aba02ac5734f admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.889147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db82ec-ca16-4d36-a347-aba02ac5734f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.891646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db82ec-ca16-4d36-a347-aba02ac5734f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.915016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 885/1777] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:34 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:34.919197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eef48-889d-4a50-b5e6-cfdef635037f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.930258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eef48-889d-4a50-b5e6-cfdef635037f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.935083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b0337-f646-4f64-85e4-810c6f00269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.935844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0337-f646-4f64-85e4-810c6f00269a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.936127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0337-f646-4f64-85e4-810c6f00269a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.936441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b0337-f646-4f64-85e4-810c6f00269a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:34.937343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b0337-f646-4f64-85e4-810c6f00269a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:34.939656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b0337-f646-4f64-85e4-810c6f00269a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:34.958848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 893/1778] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:34.961273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db82ec-ca16-4d36-a347-aba02ac5734f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:34.973176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db82ec-ca16-4d36-a347-aba02ac5734f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:34.978149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f1c00-5ec0-4abb-bb4e-4b4a513acf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:34.979297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f1c00-5ec0-4abb-bb4e-4b4a513acf47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:34.979486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f1c00-5ec0-4abb-bb4e-4b4a513acf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:34.979767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f1c00-5ec0-4abb-bb4e-4b4a513acf47 admin admin] PATH_INFO: `/v3/projects/78a798257db7482ab869cbb2af19fa50/users/3d52ae7f36564d34b826bf1b1b002ef6/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:35.006774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f1c00-5ec0-4abb-bb4e-4b4a513acf47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a798257db7482ab869cbb2af19fa50, user_id=3d52ae7f36564d34b826bf1b1b002ef6, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:35.011808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f1c00-5ec0-4abb-bb4e-4b4a513acf47 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:35.031041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 886/1779] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:34 2022] PUT /identity/v3/projects/78a798257db7482ab869cbb2af19fa50/users/3d52ae7f36564d34b826bf1b1b002ef6/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:35.036468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b0337-f646-4f64-85e4-810c6f00269a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:35.051681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b0337-f646-4f64-85e4-810c6f00269a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:35.115269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f038be-ff3c-4b65-b1bc-6f7f92d5c717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:35.115977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f038be-ff3c-4b65-b1bc-6f7f92d5c717 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:35.116312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f038be-ff3c-4b65-b1bc-6f7f92d5c717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:35.116609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f038be-ff3c-4b65-b1bc-6f7f92d5c717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:35.117359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f038be-ff3c-4b65-b1bc-6f7f92d5c717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:35.119056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f038be-ff3c-4b65-b1bc-6f7f92d5c717 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:35.130543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 894/1780] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:35.135181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f1c00-5ec0-4abb-bb4e-4b4a513acf47 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:35.147233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f1c00-5ec0-4abb-bb4e-4b4a513acf47 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:35.152778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:35.154173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:35.154173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:35.154377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] PATH_INFO: `/v3/projects/78a798257db7482ab869cbb2af19fa50/users/3d52ae7f36564d34b826bf1b1b002ef6/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:35.156525 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a798257db7482ab869cbb2af19fa50, user_id=3d52ae7f36564d34b826bf1b1b002ef6, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:35.159798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:35.172256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 887/1781] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:35 2022] PUT /identity/v3/projects/78a798257db7482ab869cbb2af19fa50/users/3d52ae7f36564d34b826bf1b1b002ef6/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:35.182657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dc05b-ec7d-4547-833f-dead71efe8cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:35.182959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dc05b-ec7d-4547-833f-dead71efe8cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:35.183253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dc05b-ec7d-4547-833f-dead71efe8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:35.209061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:35.212890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-968dc05b-ec7d-4547-833f-dead71efe8cb None None] MFA Rules not processed for user `3d52ae7f36564d34b826bf1b1b002ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:35.214525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-968dc05b-ec7d-4547-833f-dead71efe8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:35.328483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 895/1782] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:35 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:35.341209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:35.342911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:35.413780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b465f-c945-4cf0-9f2a-67cc48a62ff1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:35.417485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:35.418195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:35.418484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:35.418769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:35.429226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:35.430720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:35.441170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 888/1783] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:35 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:35.985328 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968dc05b-ec7d-4547-833f-dead71efe8cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:36.087027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:36.088622 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:36.117625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968dc05b-ec7d-4547-833f-dead71efe8cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:36.167021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:36.167940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:36.168338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:36.168613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] PATH_INFO: `/v3/users/6567d66806aa4ee1b76e9723fd24c1fa` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:36.171669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6567d66806aa4ee1b76e9723fd24c1fa)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:36.174523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:36.322504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa42973-c0ca-45fc-a630-a129db7de5da service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:36.339141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64b733-0260-42b7-8d6d-6150df740f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:36.339890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64b733-0260-42b7-8d6d-6150df740f49 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:36.340285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64b733-0260-42b7-8d6d-6150df740f49 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:36.340787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64b733-0260-42b7-8d6d-6150df740f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:36.344403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1d64b733-0260-42b7-8d6d-6150df740f49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:36.367193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64b733-0260-42b7-8d6d-6150df740f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:36.369318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64b733-0260-42b7-8d6d-6150df740f49 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:36.388462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6567d66806aa4ee1b76e9723fd24c1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:36.411344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6567d66806aa4ee1b76e9723fd24c1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:36.412526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 889/1784] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:36.413348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6567d66806aa4ee1b76e9723fd24c1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:36.414561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6567d66806aa4ee1b76e9723fd24c1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:36.424751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e2d6bc7-507d-40b7-8b38-ba535f20b96c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6567d66806aa4ee1b76e9723fd24c1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:36.470567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 896/1785] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:35 2022] DELETE /identity/v3/users/6567d66806aa4ee1b76e9723fd24c1fa => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:36.472596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054e916-e850-471d-9108-b7f076485f44 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:36.472953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054e916-e850-471d-9108-b7f076485f44 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:36.473240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054e916-e850-471d-9108-b7f076485f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:36.504853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9054e916-e850-471d-9108-b7f076485f44 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:36.507248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9054e916-e850-471d-9108-b7f076485f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:36.625823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:36.698370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 890/1786] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:36.837852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27720b-6b08-43e6-871d-002b9d7fed5e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:36.838146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27720b-6b08-43e6-871d-002b9d7fed5e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:36.838515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27720b-6b08-43e6-871d-002b9d7fed5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:36.847576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054e916-e850-471d-9108-b7f076485f44 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:36.848850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9054e916-e850-471d-9108-b7f076485f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:36.889275 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cf27720b-6b08-43e6-871d-002b9d7fed5e None None] MFA Rules not processed for user `166b42a5d6c04659afb20c088dfead76`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:36.906813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cf27720b-6b08-43e6-871d-002b9d7fed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:36.938095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:36.994246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054e916-e850-471d-9108-b7f076485f44 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:36.999527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:37.000938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:37.002006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:37.002454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:37.016034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:37.019287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:37.034034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 891/1787] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:37.041417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 897/1788] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:38.605623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:38.607262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:38.627016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a516ac-3c0c-4240-84ef-c555dcec2dd5 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:38.645367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:38.646350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:38.647068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:38.647425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] PATH_INFO: `/v3/projects/5871b927396d4feead42cd97beaa7ec4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:38.649891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5871b927396d4feead42cd97beaa7ec4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:38.652095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:38.797668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5871b927396d4feead42cd97beaa7ec4', 'request_id': 'req-632cd2f2-b91e-4dde-a8ba-9aad10281f42'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:38.809123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5871b927396d4feead42cd97beaa7ec4', 'request_id': 'req-632cd2f2-b91e-4dde-a8ba-9aad10281f42'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:38.824827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5871b927396d4feead42cd97beaa7ec4', 'request_id': 'req-632cd2f2-b91e-4dde-a8ba-9aad10281f42'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:38.846623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] The token cache is being invalidate because project 5871b927396d4feead42cd97beaa7ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:38.846931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:38.847827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:38.852816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 892/1789] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:38 2022] DELETE /identity/v3/projects/5871b927396d4feead42cd97beaa7ec4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:38.854060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27720b-6b08-43e6-871d-002b9d7fed5e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:38.877963 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27720b-6b08-43e6-871d-002b9d7fed5e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:39.031705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:39.034098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:39.034417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:39.034746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] PATH_INFO: `/v3/users/3d52ae7f36564d34b826bf1b1b002ef6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:39.038278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d52ae7f36564d34b826bf1b1b002ef6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:39.040304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:39.140554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d52ae7f36564d34b826bf1b1b002ef6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.171342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d52ae7f36564d34b826bf1b1b002ef6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.172337 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d52ae7f36564d34b826bf1b1b002ef6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.177775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d52ae7f36564d34b826bf1b1b002ef6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.189428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-167a7505-bc63-479f-a52a-9cda96a6090f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d52ae7f36564d34b826bf1b1b002ef6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.235535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 898/1790] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:38 2022] DELETE /identity/v3/users/3d52ae7f36564d34b826bf1b1b002ef6 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:39.703947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:39.706173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:39.722177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cd2f2-b91e-4dde-a8ba-9aad10281f42 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:39.813443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:39.814534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:39.815052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:39.815369 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] PATH_INFO: `/v3/projects/78a798257db7482ab869cbb2af19fa50` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:39.817501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a798257db7482ab869cbb2af19fa50)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:39.819447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:39.822113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:39.879007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a798257db7482ab869cbb2af19fa50', 'request_id': 'req-90b0c5ee-e191-4677-9c2f-892459dc44df'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.887795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a798257db7482ab869cbb2af19fa50', 'request_id': 'req-90b0c5ee-e191-4677-9c2f-892459dc44df'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.896918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a798257db7482ab869cbb2af19fa50', 'request_id': 'req-90b0c5ee-e191-4677-9c2f-892459dc44df'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.914745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] The token cache is being invalidate because project 78a798257db7482ab869cbb2af19fa50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:39.915150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.917811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-90b0c5ee-e191-4677-9c2f-892459dc44df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:39.919737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 893/1791] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:39 2022] DELETE /identity/v3/projects/78a798257db7482ab869cbb2af19fa50 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:39.925994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b92db-94ad-4470-8412-7ced30b034ff None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:39.926505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b92db-94ad-4470-8412-7ced30b034ff None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:39.926600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b92db-94ad-4470-8412-7ced30b034ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:39.949421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6b6b92db-94ad-4470-8412-7ced30b034ff None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:39.950659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6b6b92db-94ad-4470-8412-7ced30b034ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:39.993989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:40.018306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 899/1792] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:40.026946 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae100af-8e29-4092-931a-4f0ca2be76bb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.027616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae100af-8e29-4092-931a-4f0ca2be76bb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.027903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae100af-8e29-4092-931a-4f0ca2be76bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.064912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fae100af-8e29-4092-931a-4f0ca2be76bb None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:40.066169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fae100af-8e29-4092-931a-4f0ca2be76bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:40.075695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 894/1793] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:40.080462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b92db-94ad-4470-8412-7ced30b034ff None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.095001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b92db-94ad-4470-8412-7ced30b034ff None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.100993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.101766 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.102470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.102470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.103056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:40.103365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.105226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.122927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 900/1794] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:40.125855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae100af-8e29-4092-931a-4f0ca2be76bb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.136205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae100af-8e29-4092-931a-4f0ca2be76bb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.141753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdd89b-4db2-4fc3-a6cf-14fbf4c1e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.142919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdd89b-4db2-4fc3-a6cf-14fbf4c1e375 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.143194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdd89b-4db2-4fc3-a6cf-14fbf4c1e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.143683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdd89b-4db2-4fc3-a6cf-14fbf4c1e375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.144413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdd89b-4db2-4fc3-a6cf-14fbf4c1e375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.146611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdd89b-4db2-4fc3-a6cf-14fbf4c1e375 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.181014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 895/1795] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:40 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:40.186422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.204272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b94a2-bd38-4626-a9f4-71e48d6a0745 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.271588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0debe7c-c87d-4167-930f-f37562ebd99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.272298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0debe7c-c87d-4167-930f-f37562ebd99a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.272589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0debe7c-c87d-4167-930f-f37562ebd99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.272868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0debe7c-c87d-4167-930f-f37562ebd99a admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.273561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0debe7c-c87d-4167-930f-f37562ebd99a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.275485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0debe7c-c87d-4167-930f-f37562ebd99a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.296248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 901/1796] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:40 2022] POST /identity/v3/users => generated 305 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:40.301006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdd89b-4db2-4fc3-a6cf-14fbf4c1e375 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.317048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdd89b-4db2-4fc3-a6cf-14fbf4c1e375 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.324652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7734a08-6335-442e-8c96-619cc47895ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.325666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7734a08-6335-442e-8c96-619cc47895ba admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.326046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7734a08-6335-442e-8c96-619cc47895ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.326327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7734a08-6335-442e-8c96-619cc47895ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.327053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7734a08-6335-442e-8c96-619cc47895ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.328880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7734a08-6335-442e-8c96-619cc47895ba admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.344593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 896/1797] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:40.349214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0debe7c-c87d-4167-930f-f37562ebd99a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.359139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0debe7c-c87d-4167-930f-f37562ebd99a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.363850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2be493-2d42-481c-9f81-882e7b6f97e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.364559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2be493-2d42-481c-9f81-882e7b6f97e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.364850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2be493-2d42-481c-9f81-882e7b6f97e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.365121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2be493-2d42-481c-9f81-882e7b6f97e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.365850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2be493-2d42-481c-9f81-882e7b6f97e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.367459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2be493-2d42-481c-9f81-882e7b6f97e8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.377744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 902/1798] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:40.381859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7734a08-6335-442e-8c96-619cc47895ba admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.393289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7734a08-6335-442e-8c96-619cc47895ba admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.398250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c19ea6-10ed-4bc4-9283-413c78a0330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.399139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c19ea6-10ed-4bc4-9283-413c78a0330b admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.399435 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c19ea6-10ed-4bc4-9283-413c78a0330b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.399706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c19ea6-10ed-4bc4-9283-413c78a0330b admin admin] PATH_INFO: `/v3/projects/f1b378a636354281a859eb90afd2fe50/users/5c3c710748864273ac4f730ed52a2389/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.415290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c19ea6-10ed-4bc4-9283-413c78a0330b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b378a636354281a859eb90afd2fe50, user_id=5c3c710748864273ac4f730ed52a2389, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.418077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c19ea6-10ed-4bc4-9283-413c78a0330b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.431403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 897/1799] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:40 2022] PUT /identity/v3/projects/f1b378a636354281a859eb90afd2fe50/users/5c3c710748864273ac4f730ed52a2389/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:40.442261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e217ff-2796-48ee-a8ce-525d8fbe318e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.442569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e217ff-2796-48ee-a8ce-525d8fbe318e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.442858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e217ff-2796-48ee-a8ce-525d8fbe318e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.470041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-60e217ff-2796-48ee-a8ce-525d8fbe318e None None] MFA Rules not processed for user `5c3c710748864273ac4f730ed52a2389`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:40.471298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-60e217ff-2796-48ee-a8ce-525d8fbe318e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:40.541828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 903/1800] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:40.546324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c19ea6-10ed-4bc4-9283-413c78a0330b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.614634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c19ea6-10ed-4bc4-9283-413c78a0330b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.620713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc337b8-bc57-4e37-a3ca-7b43dfdb24b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.621452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc337b8-bc57-4e37-a3ca-7b43dfdb24b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.621823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc337b8-bc57-4e37-a3ca-7b43dfdb24b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.622104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc337b8-bc57-4e37-a3ca-7b43dfdb24b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.622998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc337b8-bc57-4e37-a3ca-7b43dfdb24b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.624788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc337b8-bc57-4e37-a3ca-7b43dfdb24b6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.663300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 898/1801] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:40 2022] POST /identity/v3/projects => generated 409 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:40.667605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e217ff-2796-48ee-a8ce-525d8fbe318e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.681958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e217ff-2796-48ee-a8ce-525d8fbe318e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.716069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:40.749097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8603468-a462-4542-be17-1c97d73584a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.749811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603468-a462-4542-be17-1c97d73584a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.750103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603468-a462-4542-be17-1c97d73584a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.750848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603468-a462-4542-be17-1c97d73584a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.752253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8603468-a462-4542-be17-1c97d73584a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.754239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8603468-a462-4542-be17-1c97d73584a1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.776245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 904/1802] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:40 2022] POST /identity/v3/users => generated 311 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:40.780199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc337b8-bc57-4e37-a3ca-7b43dfdb24b6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.791254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc337b8-bc57-4e37-a3ca-7b43dfdb24b6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.796449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe1bf0-dc3a-49fb-97df-7987e0acdb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.797183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe1bf0-dc3a-49fb-97df-7987e0acdb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.797472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe1bf0-dc3a-49fb-97df-7987e0acdb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.797750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe1bf0-dc3a-49fb-97df-7987e0acdb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.799015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe1bf0-dc3a-49fb-97df-7987e0acdb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.800680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe1bf0-dc3a-49fb-97df-7987e0acdb7f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.811565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 899/1803] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:40.816662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8603468-a462-4542-be17-1c97d73584a1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.826866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8603468-a462-4542-be17-1c97d73584a1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.831649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a3f28-b395-4ed2-a011-3fc7bc1d1ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.832570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a3f28-b395-4ed2-a011-3fc7bc1d1ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.832866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a3f28-b395-4ed2-a011-3fc7bc1d1ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.833146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a3f28-b395-4ed2-a011-3fc7bc1d1ccb admin admin] PATH_INFO: `/v3/projects/d276cd2032b1480d9b5229c2211dd270/users/69891dd6e4df4592850bbe22e3a30e1b/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.851771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a3f28-b395-4ed2-a011-3fc7bc1d1ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d276cd2032b1480d9b5229c2211dd270, user_id=69891dd6e4df4592850bbe22e3a30e1b, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.855113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a3f28-b395-4ed2-a011-3fc7bc1d1ccb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.871516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 905/1804] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:40 2022] PUT /identity/v3/projects/d276cd2032b1480d9b5229c2211dd270/users/69891dd6e4df4592850bbe22e3a30e1b/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:40.878424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe1bf0-dc3a-49fb-97df-7987e0acdb7f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.884363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:40.896112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe1bf0-dc3a-49fb-97df-7987e0acdb7f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.962502 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76707ae8-8864-4d1e-8bc6-f1794ba4afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:40.963481 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707ae8-8864-4d1e-8bc6-f1794ba4afa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:40.963768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707ae8-8864-4d1e-8bc6-f1794ba4afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:40.964050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76707ae8-8864-4d1e-8bc6-f1794ba4afa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:40.964830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76707ae8-8864-4d1e-8bc6-f1794ba4afa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:40.966456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76707ae8-8864-4d1e-8bc6-f1794ba4afa8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:40.977820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 900/1805] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:40.981716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a3f28-b395-4ed2-a011-3fc7bc1d1ccb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:40.994879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a3f28-b395-4ed2-a011-3fc7bc1d1ccb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:40.999739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:41.000704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:41.000973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:41.001277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] PATH_INFO: `/v3/projects/d276cd2032b1480d9b5229c2211dd270/users/69891dd6e4df4592850bbe22e3a30e1b/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:41.004363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d276cd2032b1480d9b5229c2211dd270, user_id=69891dd6e4df4592850bbe22e3a30e1b, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:41.007149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:41.022223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 906/1806] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:40 2022] PUT /identity/v3/projects/d276cd2032b1480d9b5229c2211dd270/users/69891dd6e4df4592850bbe22e3a30e1b/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:41.032561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ea27-f9a3-4dee-b1d1-9bb6d776e077 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:41.032966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ea27-f9a3-4dee-b1d1-9bb6d776e077 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:41.033260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ea27-f9a3-4dee-b1d1-9bb6d776e077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:41.060399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b4b7ea27-f9a3-4dee-b1d1-9bb6d776e077 None None] MFA Rules not processed for user `69891dd6e4df4592850bbe22e3a30e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:41.061621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b4b7ea27-f9a3-4dee-b1d1-9bb6d776e077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:41.133550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 901/1807] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:41 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:41.144599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:41.146517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:41.218083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4012a-2e3e-4e8d-b7ef-76e15e2c41ab admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:41.222266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611501-4760-4697-8871-87f8219f86b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:41.222920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611501-4760-4697-8871-87f8219f86b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:41.223229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611501-4760-4697-8871-87f8219f86b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:41.223505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611501-4760-4697-8871-87f8219f86b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:41.234010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43611501-4760-4697-8871-87f8219f86b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:41.235451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43611501-4760-4697-8871-87f8219f86b0 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:41.247911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 907/1808] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:41 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:41.435320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7ea27-f9a3-4dee-b1d1-9bb6d776e077 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:41.445354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7ea27-f9a3-4dee-b1d1-9bb6d776e077 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:41.449066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f961e-724e-4586-abd3-7720d019420c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:41.449722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f961e-724e-4586-abd3-7720d019420c service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:41.450010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f961e-724e-4586-abd3-7720d019420c service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:41.450293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f961e-724e-4586-abd3-7720d019420c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:41.509165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f961e-724e-4586-abd3-7720d019420c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:41.510625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f961e-724e-4586-abd3-7720d019420c service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:41.520626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 902/1809] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:44.672840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285941-3479-4f61-abe5-09bf3218114c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:44.674408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285941-3479-4f61-abe5-09bf3218114c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:44.674408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285941-3479-4f61-abe5-09bf3218114c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:44.676709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 908/1810] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 12:04:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:04:44.753339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afe09-d651-4a56-8556-d08a15c34522 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:44.753812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afe09-d651-4a56-8556-d08a15c34522 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:44.753898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afe09-d651-4a56-8556-d08a15c34522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:44.790910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8a0afe09-d651-4a56-8556-d08a15c34522 None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:44.792278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a0afe09-d651-4a56-8556-d08a15c34522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:44.821906 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:44.948399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 903/1811] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 12:04:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:45.052753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29285941-3479-4f61-abe5-09bf3218114c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:45.054853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-29285941-3479-4f61-abe5-09bf3218114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:45.076622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29285941-3479-4f61-abe5-09bf3218114c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:45.085103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8953782-2f0c-4453-8c9c-afe537b4e83b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:45.085786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8953782-2f0c-4453-8c9c-afe537b4e83b service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:45.086078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8953782-2f0c-4453-8c9c-afe537b4e83b service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:45.088015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8953782-2f0c-4453-8c9c-afe537b4e83b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:45.098427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8953782-2f0c-4453-8c9c-afe537b4e83b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:45.100030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8953782-2f0c-4453-8c9c-afe537b4e83b service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:45.112089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 909/1812] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:49.665790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0afe09-d651-4a56-8556-d08a15c34522 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:49.672669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a0afe09-d651-4a56-8556-d08a15c34522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:49.779792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0afe09-d651-4a56-8556-d08a15c34522 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:49.784642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:49.785553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:49.785841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:49.786154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] PATH_INFO: `/v3/users/d1c76291c9f94809ba2210e8954343aa` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:49.787450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c76291c9f94809ba2210e8954343aa)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:49.789640 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:49.857577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c76291c9f94809ba2210e8954343aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:49.874639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c76291c9f94809ba2210e8954343aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:49.875940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c76291c9f94809ba2210e8954343aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:49.888444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c76291c9f94809ba2210e8954343aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:49.889754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c76291c9f94809ba2210e8954343aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:49.915647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 904/1813] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:49 2022] DELETE /identity/v3/users/d1c76291c9f94809ba2210e8954343aa => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:49.917634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478ad6-e7ea-4202-9927-6cc8695c9f33 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:49.918157 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478ad6-e7ea-4202-9927-6cc8695c9f33 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:49.918250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478ad6-e7ea-4202-9927-6cc8695c9f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:49.941129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-79478ad6-e7ea-4202-9927-6cc8695c9f33 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:49.942518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-79478ad6-e7ea-4202-9927-6cc8695c9f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:50.065408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 910/1814] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:50.130615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:50.132496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:50.199143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:50.245300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b3785-606c-47db-93fa-c5ff8aafc1cf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:50.249459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dd865-688e-4a62-8479-1ff4ccff7b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:50.250268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dd865-688e-4a62-8479-1ff4ccff7b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:50.250559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dd865-688e-4a62-8479-1ff4ccff7b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:50.250833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dd865-688e-4a62-8479-1ff4ccff7b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:50.278571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745dd865-688e-4a62-8479-1ff4ccff7b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:50.280056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745dd865-688e-4a62-8479-1ff4ccff7b46 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:50.308274 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 905/1815] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:50.659722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aecab7-be27-4af9-af5c-0056d7f7e7db None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:50.660016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aecab7-be27-4af9-af5c-0056d7f7e7db None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:50.660333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aecab7-be27-4af9-af5c-0056d7f7e7db None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:50.665470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 911/1816] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 12:04:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:04:50.675100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679210-e0c1-4359-8e45-e38ad4f5571c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:50.675634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679210-e0c1-4359-8e45-e38ad4f5571c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:50.675893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36679210-e0c1-4359-8e45-e38ad4f5571c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:50.717346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-36679210-e0c1-4359-8e45-e38ad4f5571c None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:50.723538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-36679210-e0c1-4359-8e45-e38ad4f5571c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:50.859593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aecab7-be27-4af9-af5c-0056d7f7e7db None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:50.868575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-03aecab7-be27-4af9-af5c-0056d7f7e7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:50.899849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aecab7-be27-4af9-af5c-0056d7f7e7db None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:50.905360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:50.909319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:50.909856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:50.910240 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] PATH_INFO: `/v3/users/5c3c710748864273ac4f730ed52a2389` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:50.911697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3c710748864273ac4f730ed52a2389)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:50.917160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:50.952440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 906/1817] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 12:04:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:50.952812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:51.028762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3c710748864273ac4f730ed52a2389'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.053482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3c710748864273ac4f730ed52a2389'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.054443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3c710748864273ac4f730ed52a2389'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.055237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3c710748864273ac4f730ed52a2389'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.081612 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3c710748864273ac4f730ed52a2389'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.101845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 912/1818] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:50 2022] DELETE /identity/v3/users/5c3c710748864273ac4f730ed52a2389 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:51.107154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fde9-6e57-47f6-981e-994ae71db4de None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:51.107410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fde9-6e57-47f6-981e-994ae71db4de None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:51.108559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8fde9-6e57-47f6-981e-994ae71db4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:51.133876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-91d8fde9-6e57-47f6-981e-994ae71db4de None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:51.135291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-91d8fde9-6e57-47f6-981e-994ae71db4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:51.242642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 907/1819] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:51.276355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:51.277843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:51.321824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8fde9-6e57-47f6-981e-994ae71db4de None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:51.323335 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-91d8fde9-6e57-47f6-981e-994ae71db4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:51.398691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8553-5b70-43ef-9a92-9cfdf0d60afa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:51.433095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:51.434682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:51.435196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:51.436212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:51.441113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8fde9-6e57-47f6-981e-994ae71db4de None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:51.452797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:51.455822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:51.460709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:51.461638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:51.462109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:51.462570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:51.470434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 913/1820] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:51.498093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:51.523521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:51.539365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:51.543868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:51.616761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 908/1821] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:51.811266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:51.812871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:51.837508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481cde6-cdee-40d4-8838-61ae3c5a54e1 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:51.843751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:51.845532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:51.846000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:51.846458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] PATH_INFO: `/v3/projects/1937b04b46c440018587024189bc0ed7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:51.848792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1937b04b46c440018587024189bc0ed7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:51.851213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:51.951680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1937b04b46c440018587024189bc0ed7', 'request_id': 'req-7cc3a15a-a9b3-428f-ad65-75ecd319316b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.959860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1937b04b46c440018587024189bc0ed7', 'request_id': 'req-7cc3a15a-a9b3-428f-ad65-75ecd319316b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.966669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1937b04b46c440018587024189bc0ed7', 'request_id': 'req-7cc3a15a-a9b3-428f-ad65-75ecd319316b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.984408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] The token cache is being invalidate because project 1937b04b46c440018587024189bc0ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:51.984725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.985926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:51.991256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 914/1822] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:51 2022] DELETE /identity/v3/projects/1937b04b46c440018587024189bc0ed7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:51.992489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:52.004949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84372eb-3f24-484f-9f55-e8a82f85ab86 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:52.069139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:52.070437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:52.070877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:52.071429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] PATH_INFO: `/v3/users/7b2d548d83e140dbaa96792159b626e6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:52.073244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2d548d83e140dbaa96792159b626e6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:52.075594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:52.173206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2d548d83e140dbaa96792159b626e6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.189688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2d548d83e140dbaa96792159b626e6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.190342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2d548d83e140dbaa96792159b626e6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.198629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2d548d83e140dbaa96792159b626e6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.199654 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2d548d83e140dbaa96792159b626e6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.221769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 909/1823] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:51 2022] DELETE /identity/v3/users/7b2d548d83e140dbaa96792159b626e6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:52.398044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:52.399848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:52.422560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3a15a-a9b3-428f-ad65-75ecd319316b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:52.443066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:52.499264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:52.500139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:52.500543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:52.502121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] PATH_INFO: `/v3/projects/f1b378a636354281a859eb90afd2fe50` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:52.504882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b378a636354281a859eb90afd2fe50)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:52.506858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:52.638060 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b378a636354281a859eb90afd2fe50', 'request_id': 'req-0708a031-1bb6-468c-98d1-9e63d29287ef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.650696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b378a636354281a859eb90afd2fe50', 'request_id': 'req-0708a031-1bb6-468c-98d1-9e63d29287ef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.662207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b378a636354281a859eb90afd2fe50', 'request_id': 'req-0708a031-1bb6-468c-98d1-9e63d29287ef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.678174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] The token cache is being invalidate because project f1b378a636354281a859eb90afd2fe50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:52.678689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.679802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.682069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 915/1824] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:52 2022] DELETE /identity/v3/projects/f1b378a636354281a859eb90afd2fe50 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:52.685896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:52.687702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:52.732456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816b9dc-b0a8-4e8a-9476-1ec37b9abd8c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:52.738315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:52.739718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:52.855885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:52.855885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:52.855885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:52.855885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] PATH_INFO: `/v3/users/69891dd6e4df4592850bbe22e3a30e1b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:52.857071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=69891dd6e4df4592850bbe22e3a30e1b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:52.859330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:52.883575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a031-1bb6-468c-98d1-9e63d29287ef admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:52.889348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:52.891717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:52.894256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:52.894700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:52.895445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] PATH_INFO: `/v3/projects/9fa6bcce0fe6436884a274f2961b3526` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:52.897001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa6bcce0fe6436884a274f2961b3526)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:52.899743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:52.946234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69891dd6e4df4592850bbe22e3a30e1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.961415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa6bcce0fe6436884a274f2961b3526', 'request_id': 'req-c985fe14-9c8c-468a-8daf-3749c56f9d23'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.963216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:52.963545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69891dd6e4df4592850bbe22e3a30e1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.964379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69891dd6e4df4592850bbe22e3a30e1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.973763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa6bcce0fe6436884a274f2961b3526', 'request_id': 'req-c985fe14-9c8c-468a-8daf-3749c56f9d23'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.981059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69891dd6e4df4592850bbe22e3a30e1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.981232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa6bcce0fe6436884a274f2961b3526', 'request_id': 'req-c985fe14-9c8c-468a-8daf-3749c56f9d23'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.982063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-49e97da1-c3ee-449e-b73d-653de95fb76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69891dd6e4df4592850bbe22e3a30e1b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:52.995944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] The token cache is being invalidate because project 9fa6bcce0fe6436884a274f2961b3526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:52.996492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:53.000129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 910/1825] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:52 2022] DELETE /identity/v3/users/69891dd6e4df4592850bbe22e3a30e1b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:53.004893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c985fe14-9c8c-468a-8daf-3749c56f9d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:53.011985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 916/1826] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:52 2022] DELETE /identity/v3/projects/9fa6bcce0fe6436884a274f2961b3526 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:53.024043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b75481-a86a-414b-beb2-620c4be45642 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.024368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b75481-a86a-414b-beb2-620c4be45642 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.024707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b75481-a86a-414b-beb2-620c4be45642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.049593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c9b75481-a86a-414b-beb2-620c4be45642 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:53.051094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c9b75481-a86a-414b-beb2-620c4be45642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:53.127847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 911/1827] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:53.135975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833b2a-79bf-4682-8f37-941b96d31132 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.136654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833b2a-79bf-4682-8f37-941b96d31132 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.137132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833b2a-79bf-4682-8f37-941b96d31132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.168074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-56833b2a-79bf-4682-8f37-941b96d31132 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:53.169324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-56833b2a-79bf-4682-8f37-941b96d31132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:53.181437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 917/1828] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:53.182896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b75481-a86a-414b-beb2-620c4be45642 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.197803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b75481-a86a-414b-beb2-620c4be45642 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.210519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.211216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.211510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.212445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.212642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:53.213072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.214876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.240240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 912/1829] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:53.244322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56833b2a-79bf-4682-8f37-941b96d31132 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.258317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56833b2a-79bf-4682-8f37-941b96d31132 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.263443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a955ae-979b-437e-8839-5fa955d0f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.264127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a955ae-979b-437e-8839-5fa955d0f064 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.264468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a955ae-979b-437e-8839-5fa955d0f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.264748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a955ae-979b-437e-8839-5fa955d0f064 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.265452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a955ae-979b-437e-8839-5fa955d0f064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.267572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a955ae-979b-437e-8839-5fa955d0f064 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.300610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 918/1830] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:53.306800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.320336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb1dbc-dcb3-4f11-9204-baa5995aeca1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.399675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdf690-aaf5-46cf-b6d7-2566370079fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.400394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf690-aaf5-46cf-b6d7-2566370079fe admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.401124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf690-aaf5-46cf-b6d7-2566370079fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.401124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdf690-aaf5-46cf-b6d7-2566370079fe admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.401658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdf690-aaf5-46cf-b6d7-2566370079fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.403639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdf690-aaf5-46cf-b6d7-2566370079fe admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.430933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 913/1831] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:53.435119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a955ae-979b-437e-8839-5fa955d0f064 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.447441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a955ae-979b-437e-8839-5fa955d0f064 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.452392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad098f97-e994-492b-9906-0031a4e9002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.453018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad098f97-e994-492b-9906-0031a4e9002d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.453312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad098f97-e994-492b-9906-0031a4e9002d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.453601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad098f97-e994-492b-9906-0031a4e9002d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.455106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad098f97-e994-492b-9906-0031a4e9002d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.456778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad098f97-e994-492b-9906-0031a4e9002d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.473168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 919/1832] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:53.474791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdf690-aaf5-46cf-b6d7-2566370079fe admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.487599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdf690-aaf5-46cf-b6d7-2566370079fe admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.492823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.493534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.493827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.494100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.495139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.497624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.501386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:53.514283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 914/1833] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:53.518514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad098f97-e994-492b-9906-0031a4e9002d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.530421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad098f97-e994-492b-9906-0031a4e9002d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.546930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdcfe8-6199-4f48-9ca4-3c83ed445263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.548656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdcfe8-6199-4f48-9ca4-3c83ed445263 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.549005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdcfe8-6199-4f48-9ca4-3c83ed445263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.549297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdcfe8-6199-4f48-9ca4-3c83ed445263 admin admin] PATH_INFO: `/v3/projects/2d37e17fe95949f291de5394c3697ae9/users/2bf3bc6ce197486eb02bf50eee7a148b/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.560964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.562713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:53.573486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdcfe8-6199-4f48-9ca4-3c83ed445263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d37e17fe95949f291de5394c3697ae9, user_id=2bf3bc6ce197486eb02bf50eee7a148b, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.576625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdcfe8-6199-4f48-9ca4-3c83ed445263 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.582338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fcb98-9a61-48ce-afde-becbfdae0b7d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.590592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.591514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.591802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.592081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] PATH_INFO: `/v3/projects/d276cd2032b1480d9b5229c2211dd270` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.593644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d276cd2032b1480d9b5229c2211dd270)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.595719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.596222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 920/1834] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:53 2022] PUT /identity/v3/projects/2d37e17fe95949f291de5394c3697ae9/users/2bf3bc6ce197486eb02bf50eee7a148b/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:53.607368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f04ea-e45c-4ab3-9fa9-9b942031167a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.607660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f04ea-e45c-4ab3-9fa9-9b942031167a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.607953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f04ea-e45c-4ab3-9fa9-9b942031167a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.653110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd276cd2032b1480d9b5229c2211dd270', 'request_id': 'req-8fec25dc-72a6-43df-8528-7d2d4e4f8657'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:53.661385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd276cd2032b1480d9b5229c2211dd270', 'request_id': 'req-8fec25dc-72a6-43df-8528-7d2d4e4f8657'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:53.661642 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-197f04ea-e45c-4ab3-9fa9-9b942031167a None None] MFA Rules not processed for user `2bf3bc6ce197486eb02bf50eee7a148b`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:53.662868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-197f04ea-e45c-4ab3-9fa9-9b942031167a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:53.669391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd276cd2032b1480d9b5229c2211dd270', 'request_id': 'req-8fec25dc-72a6-43df-8528-7d2d4e4f8657'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:53.686653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] The token cache is being invalidate because project d276cd2032b1480d9b5229c2211dd270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:53.687124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:53.688390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8fec25dc-72a6-43df-8528-7d2d4e4f8657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:53.694860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 915/1835] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:53 2022] DELETE /identity/v3/projects/d276cd2032b1480d9b5229c2211dd270 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:04:53.700096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23449523-25c8-4762-aeea-60f115b0cd15 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.702834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23449523-25c8-4762-aeea-60f115b0cd15 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.703497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23449523-25c8-4762-aeea-60f115b0cd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.730873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-23449523-25c8-4762-aeea-60f115b0cd15 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:53.732578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-23449523-25c8-4762-aeea-60f115b0cd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:53.757451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 921/1836] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:53.762153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f04ea-e45c-4ab3-9fa9-9b942031167a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.763641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-197f04ea-e45c-4ab3-9fa9-9b942031167a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:53.815602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 916/1837] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:53.823387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b9440-cf40-4f67-ab6e-2df318a80627 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.824315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b9440-cf40-4f67-ab6e-2df318a80627 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.824773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b9440-cf40-4f67-ab6e-2df318a80627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.849898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9b4b9440-cf40-4f67-ab6e-2df318a80627 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:53.851396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9b4b9440-cf40-4f67-ab6e-2df318a80627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:53.864569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 917/1838] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:53.866497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b9440-cf40-4f67-ab6e-2df318a80627 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.875778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f04ea-e45c-4ab3-9fa9-9b942031167a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.881773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b9440-cf40-4f67-ab6e-2df318a80627 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.885092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca8121-9d1c-4d9b-b8a5-1243f005bd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.885979 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca8121-9d1c-4d9b-b8a5-1243f005bd63 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.886407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca8121-9d1c-4d9b-b8a5-1243f005bd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.886869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca8121-9d1c-4d9b-b8a5-1243f005bd63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.887975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca8121-9d1c-4d9b-b8a5-1243f005bd63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.888615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.889456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.889965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.890310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.891092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca8121-9d1c-4d9b-b8a5-1243f005bd63 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.891198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:53.892070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.894653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:53.908837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:53.940197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 918/1839] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:53.941149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.952101 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80e423-883d-4668-bd63-d78c2cbb596e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:53.958435 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfcbed-97de-4508-b954-c08bfb9ac8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:53.959764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 922/1840] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:04:53.960706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcbed-97de-4508-b954-c08bfb9ac8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:53.962680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcbed-97de-4508-b954-c08bfb9ac8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:53.963054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcbed-97de-4508-b954-c08bfb9ac8e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:53.963862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfcbed-97de-4508-b954-c08bfb9ac8e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:53.963970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca8121-9d1c-4d9b-b8a5-1243f005bd63 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:53.971899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfcbed-97de-4508-b954-c08bfb9ac8e1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.024854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 919/1841] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:54.029217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfcbed-97de-4508-b954-c08bfb9ac8e1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.060566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca8121-9d1c-4d9b-b8a5-1243f005bd63 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.098013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:54.123111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfcbed-97de-4508-b954-c08bfb9ac8e1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.128189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101ee32-1649-4686-950b-74342dff85e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.128727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f377-e636-4857-a196-03c6ec47db56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.128921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101ee32-1649-4686-950b-74342dff85e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.129184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101ee32-1649-4686-950b-74342dff85e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.129430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f377-e636-4857-a196-03c6ec47db56 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.129616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101ee32-1649-4686-950b-74342dff85e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.129752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f377-e636-4857-a196-03c6ec47db56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.130022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f377-e636-4857-a196-03c6ec47db56 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.130190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101ee32-1649-4686-950b-74342dff85e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.130882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358f377-e636-4857-a196-03c6ec47db56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.132077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101ee32-1649-4686-950b-74342dff85e9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.132905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358f377-e636-4857-a196-03c6ec47db56 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.159518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 920/1842] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:54 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:54.160603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 923/1843] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:53 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:04:54.161063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f377-e636-4857-a196-03c6ec47db56 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.164370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101ee32-1649-4686-950b-74342dff85e9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.173373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f377-e636-4857-a196-03c6ec47db56 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.177661 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101ee32-1649-4686-950b-74342dff85e9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.178163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402287d7-0315-4b81-a430-78988ff344f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.178850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402287d7-0315-4b81-a430-78988ff344f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.179135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402287d7-0315-4b81-a430-78988ff344f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.179410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402287d7-0315-4b81-a430-78988ff344f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.180234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402287d7-0315-4b81-a430-78988ff344f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.181916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402287d7-0315-4b81-a430-78988ff344f8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.182947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be59025-9f49-4e1a-9d82-fc26ab50fb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.183855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59025-9f49-4e1a-9d82-fc26ab50fb78 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.184366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59025-9f49-4e1a-9d82-fc26ab50fb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.184816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59025-9f49-4e1a-9d82-fc26ab50fb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.185566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be59025-9f49-4e1a-9d82-fc26ab50fb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.187179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be59025-9f49-4e1a-9d82-fc26ab50fb78 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.196607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 921/1844] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:54.197811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402287d7-0315-4b81-a430-78988ff344f8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.214216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 924/1845] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:54.216556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402287d7-0315-4b81-a430-78988ff344f8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.220660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be59025-9f49-4e1a-9d82-fc26ab50fb78 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.231453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0364ad-e0fd-4a89-8343-cb55ad723e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.232790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0364ad-e0fd-4a89-8343-cb55ad723e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.233073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0364ad-e0fd-4a89-8343-cb55ad723e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.233353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0364ad-e0fd-4a89-8343-cb55ad723e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.236563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0364ad-e0fd-4a89-8343-cb55ad723e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.238733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0364ad-e0fd-4a89-8343-cb55ad723e20 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.241598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be59025-9f49-4e1a-9d82-fc26ab50fb78 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.247132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56e16a-9c95-48c1-9fd1-a1bce3d0cf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.248093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56e16a-9c95-48c1-9fd1-a1bce3d0cf81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.248447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56e16a-9c95-48c1-9fd1-a1bce3d0cf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.248733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56e16a-9c95-48c1-9fd1-a1bce3d0cf81 admin admin] PATH_INFO: `/v3/projects/b035cbab320747f79860603860f12459/users/0e7001445b8c4cd39c066f902117c4e9/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.255196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 922/1846] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:54.259628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0364ad-e0fd-4a89-8343-cb55ad723e20 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.275587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0364ad-e0fd-4a89-8343-cb55ad723e20 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.279178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56e16a-9c95-48c1-9fd1-a1bce3d0cf81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b035cbab320747f79860603860f12459, user_id=0e7001445b8c4cd39c066f902117c4e9, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.281441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60423faf-ff7e-488b-8afa-7bda17e87c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.282493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56e16a-9c95-48c1-9fd1-a1bce3d0cf81 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.283289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423faf-ff7e-488b-8afa-7bda17e87c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.283841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423faf-ff7e-488b-8afa-7bda17e87c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.284136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423faf-ff7e-488b-8afa-7bda17e87c14 admin admin] PATH_INFO: `/v3/projects/1a1116efd72f43d5ab05225c05323b15/users/08baec22d1bb41caa31c828d3ad0fb05/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.304671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 925/1847] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:54 2022] PUT /identity/v3/projects/b035cbab320747f79860603860f12459/users/0e7001445b8c4cd39c066f902117c4e9/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:54.307531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60423faf-ff7e-488b-8afa-7bda17e87c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1116efd72f43d5ab05225c05323b15, user_id=08baec22d1bb41caa31c828d3ad0fb05, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.308574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56e16a-9c95-48c1-9fd1-a1bce3d0cf81 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.311260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60423faf-ff7e-488b-8afa-7bda17e87c14 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.333733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 923/1848] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:54 2022] PUT /identity/v3/projects/1a1116efd72f43d5ab05225c05323b15/users/08baec22d1bb41caa31c828d3ad0fb05/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:54.343899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ddd3-cca1-4a6e-92e1-b6837ead958d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.344277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ddd3-cca1-4a6e-92e1-b6837ead958d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.345736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ddd3-cca1-4a6e-92e1-b6837ead958d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.400567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7c95ddd3-cca1-4a6e-92e1-b6837ead958d None None] MFA Rules not processed for user `08baec22d1bb41caa31c828d3ad0fb05`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:54.402998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7c95ddd3-cca1-4a6e-92e1-b6837ead958d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:54.416548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:54.451788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56e16a-9c95-48c1-9fd1-a1bce3d0cf81 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.593335 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 924/1849] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:54.599146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ddd3-cca1-4a6e-92e1-b6837ead958d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.633800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d662601-c448-4e39-9efc-583d46f30b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.634111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d662601-c448-4e39-9efc-583d46f30b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.634392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d662601-c448-4e39-9efc-583d46f30b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.634686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d662601-c448-4e39-9efc-583d46f30b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.635456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d662601-c448-4e39-9efc-583d46f30b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.643255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d662601-c448-4e39-9efc-583d46f30b9e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.673769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 926/1850] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:54.674916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d662601-c448-4e39-9efc-583d46f30b9e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.687181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d662601-c448-4e39-9efc-583d46f30b9e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.695610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eda101-b5c8-4565-be64-77af7338f17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.696771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eda101-b5c8-4565-be64-77af7338f17d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.697522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eda101-b5c8-4565-be64-77af7338f17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.697822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eda101-b5c8-4565-be64-77af7338f17d admin admin] PATH_INFO: `/v3/projects/b035cbab320747f79860603860f12459/users/0e7001445b8c4cd39c066f902117c4e9/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.700843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eda101-b5c8-4565-be64-77af7338f17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b035cbab320747f79860603860f12459, user_id=0e7001445b8c4cd39c066f902117c4e9, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.706134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eda101-b5c8-4565-be64-77af7338f17d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.721710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 927/1851] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:54 2022] PUT /identity/v3/projects/b035cbab320747f79860603860f12459/users/0e7001445b8c4cd39c066f902117c4e9/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:54.733730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629507-6e4c-434b-ab93-8285fa35ddfe None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.734043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629507-6e4c-434b-ab93-8285fa35ddfe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.734312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629507-6e4c-434b-ab93-8285fa35ddfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.767528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ddd3-cca1-4a6e-92e1-b6837ead958d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:54.801283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:54.811637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-80629507-6e4c-434b-ab93-8285fa35ddfe None None] MFA Rules not processed for user `0e7001445b8c4cd39c066f902117c4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:54.813050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-80629507-6e4c-434b-ab93-8285fa35ddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:54.902863 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701ae18-492d-4ac8-b95d-4f22b2057614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:54.903791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701ae18-492d-4ac8-b95d-4f22b2057614 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:54.904081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701ae18-492d-4ac8-b95d-4f22b2057614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:54.904638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701ae18-492d-4ac8-b95d-4f22b2057614 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:54.905375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701ae18-492d-4ac8-b95d-4f22b2057614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:54.907804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701ae18-492d-4ac8-b95d-4f22b2057614 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:54.952925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 925/1852] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:54 2022] POST /identity/v3/projects => generated 385 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:54.958842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701ae18-492d-4ac8-b95d-4f22b2057614 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.968628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 928/1853] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:54.977149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80629507-6e4c-434b-ab93-8285fa35ddfe None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:54.981290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-80629507-6e4c-434b-ab93-8285fa35ddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:55.098859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701ae18-492d-4ac8-b95d-4f22b2057614 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:55.111214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0e99d-f155-4de3-a4aa-d8b632b1d611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:55.112012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0e99d-f155-4de3-a4aa-d8b632b1d611 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.112322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0e99d-f155-4de3-a4aa-d8b632b1d611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.112697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0e99d-f155-4de3-a4aa-d8b632b1d611 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.113322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0e99d-f155-4de3-a4aa-d8b632b1d611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:55.115256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0e99d-f155-4de3-a4aa-d8b632b1d611 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:55.140411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80629507-6e4c-434b-ab93-8285fa35ddfe None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:55.167169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 926/1854] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:54 2022] POST /identity/v3/users => generated 299 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:04:55.171395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0e99d-f155-4de3-a4aa-d8b632b1d611 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:55.193029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:55.217350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0e99d-f155-4de3-a4aa-d8b632b1d611 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:55.237093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86a14a-cad1-4b9d-ab5d-98adbccd4f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:55.238241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86a14a-cad1-4b9d-ab5d-98adbccd4f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.238241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86a14a-cad1-4b9d-ab5d-98adbccd4f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.238241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86a14a-cad1-4b9d-ab5d-98adbccd4f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.240809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86a14a-cad1-4b9d-ab5d-98adbccd4f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:55.246584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86a14a-cad1-4b9d-ab5d-98adbccd4f49 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:55.274668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 927/1855] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:55.278939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86a14a-cad1-4b9d-ab5d-98adbccd4f49 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:55.302630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:55.304527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.305053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.305461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86a14a-cad1-4b9d-ab5d-98adbccd4f49 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:55.305771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.310734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e297b-b727-41bd-bedb-513fce5e136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:55.311658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e297b-b727-41bd-bedb-513fce5e136e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.312070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e297b-b727-41bd-bedb-513fce5e136e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.313097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e297b-b727-41bd-bedb-513fce5e136e admin admin] PATH_INFO: `/v3/projects/d352c2b703c546f9a67a644c79b10a09/users/dbcfe4ec08f14e53ae7b8a2fdb0eef32/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.338959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e297b-b727-41bd-bedb-513fce5e136e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d352c2b703c546f9a67a644c79b10a09, user_id=dbcfe4ec08f14e53ae7b8a2fdb0eef32, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:55.344941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e297b-b727-41bd-bedb-513fce5e136e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:55.382166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 928/1856] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:55 2022] PUT /identity/v3/projects/d352c2b703c546f9a67a644c79b10a09/users/dbcfe4ec08f14e53ae7b8a2fdb0eef32/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:55.386474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e297b-b727-41bd-bedb-513fce5e136e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:55.426402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:55.428643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:55.441737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 929/1857] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:55.478540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e297b-b727-41bd-bedb-513fce5e136e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:55.484730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba3aa4-ee25-4cc3-a7f5-f64d354851db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:55.491558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3aa4-ee25-4cc3-a7f5-f64d354851db admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.491558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3aa4-ee25-4cc3-a7f5-f64d354851db admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.492557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:55.493983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:55.503147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3aa4-ee25-4cc3-a7f5-f64d354851db admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.504035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba3aa4-ee25-4cc3-a7f5-f64d354851db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:55.505714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba3aa4-ee25-4cc3-a7f5-f64d354851db admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:55.507392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:55.511353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0308c3-834c-433b-b9df-5b08560ac580 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:55.525815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 929/1858] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:04:55.530482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba3aa4-ee25-4cc3-a7f5-f64d354851db admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:55.547904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba3aa4-ee25-4cc3-a7f5-f64d354851db admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:55.553314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42fe41-bb18-486d-8f34-126afcb3b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:55.554228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fe41-bb18-486d-8f34-126afcb3b410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.554517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fe41-bb18-486d-8f34-126afcb3b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.554800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fe41-bb18-486d-8f34-126afcb3b410 admin admin] PATH_INFO: `/v3/projects/d352c2b703c546f9a67a644c79b10a09/users/dbcfe4ec08f14e53ae7b8a2fdb0eef32/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.557336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fe41-bb18-486d-8f34-126afcb3b410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d352c2b703c546f9a67a644c79b10a09, user_id=dbcfe4ec08f14e53ae7b8a2fdb0eef32, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:55.559607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fe41-bb18-486d-8f34-126afcb3b410 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:55.580281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 930/1859] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:55 2022] PUT /identity/v3/projects/d352c2b703c546f9a67a644c79b10a09/users/dbcfe4ec08f14e53ae7b8a2fdb0eef32/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:04:55.586813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5292a-ece1-4ead-b23b-6967085dc647 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.587327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5292a-ece1-4ead-b23b-6967085dc647 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.587608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5292a-ece1-4ead-b23b-6967085dc647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.605429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdce18d-3811-4b47-8435-ffb7ce55ead3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:55.606173 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdce18d-3811-4b47-8435-ffb7ce55ead3 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.606541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdce18d-3811-4b47-8435-ffb7ce55ead3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.606824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdce18d-3811-4b47-8435-ffb7ce55ead3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.622992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-eac5292a-ece1-4ead-b23b-6967085dc647 None None] MFA Rules not processed for user `dbcfe4ec08f14e53ae7b8a2fdb0eef32`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:55.624340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eac5292a-ece1-4ead-b23b-6967085dc647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:55.664894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdce18d-3811-4b47-8435-ffb7ce55ead3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:55.666345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdce18d-3811-4b47-8435-ffb7ce55ead3 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:55.675888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 930/1860] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:55.698098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 931/1861] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:55.711312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdce18d-3811-4b47-8435-ffb7ce55ead3 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:55.721372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdce18d-3811-4b47-8435-ffb7ce55ead3 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:55.751641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:55.778749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba13c8-b7dd-4148-bf4d-c09ec545f66b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:55.779409 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba13c8-b7dd-4148-bf4d-c09ec545f66b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:55.779700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba13c8-b7dd-4148-bf4d-c09ec545f66b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:55.780498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba13c8-b7dd-4148-bf4d-c09ec545f66b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:55.789657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba13c8-b7dd-4148-bf4d-c09ec545f66b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:55.791129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba13c8-b7dd-4148-bf4d-c09ec545f66b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:55.803130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 931/1862] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:04:56.864898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5292a-ece1-4ead-b23b-6967085dc647 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:56.981891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5292a-ece1-4ead-b23b-6967085dc647 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:56.987447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:56.988418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:56.988713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:56.988998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] PATH_INFO: `/v3/users/08baec22d1bb41caa31c828d3ad0fb05` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:56.990545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08baec22d1bb41caa31c828d3ad0fb05)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:56.992566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:57.057401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08baec22d1bb41caa31c828d3ad0fb05'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:57.071376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08baec22d1bb41caa31c828d3ad0fb05'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:57.072246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08baec22d1bb41caa31c828d3ad0fb05'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:57.076762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:57.080445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08baec22d1bb41caa31c828d3ad0fb05'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:57.081657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08baec22d1bb41caa31c828d3ad0fb05'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:57.103477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 932/1863] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:56 2022] DELETE /identity/v3/users/08baec22d1bb41caa31c828d3ad0fb05 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:57.114522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c92f62-5829-4ca3-b20d-16b47d03617d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:57.116940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c92f62-5829-4ca3-b20d-16b47d03617d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:57.117366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c92f62-5829-4ca3-b20d-16b47d03617d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:57.161452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d7c92f62-5829-4ca3-b20d-16b47d03617d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:57.162930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d7c92f62-5829-4ca3-b20d-16b47d03617d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:57.266174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 932/1864] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:57.277704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:57.279670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:57.427125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49c9d3-f581-4375-a8dc-2b0d9ba79a02 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:57.440057 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975ac3e-f628-4876-8341-61c27d5927c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:57.440057 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975ac3e-f628-4876-8341-61c27d5927c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:57.440057 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975ac3e-f628-4876-8341-61c27d5927c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:57.440057 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975ac3e-f628-4876-8341-61c27d5927c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:57.456085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975ac3e-f628-4876-8341-61c27d5927c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:57.458103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975ac3e-f628-4876-8341-61c27d5927c3 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:57.479168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 933/1865] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:04:58.060510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c92f62-5829-4ca3-b20d-16b47d03617d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:58.061987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d7c92f62-5829-4ca3-b20d-16b47d03617d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:58.097160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c92f62-5829-4ca3-b20d-16b47d03617d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:58.102928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:58.103841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:58.104185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:58.104545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] PATH_INFO: `/v3/projects/1a1116efd72f43d5ab05225c05323b15` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:58.106002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1116efd72f43d5ab05225c05323b15)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:58.108227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:58.202120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1116efd72f43d5ab05225c05323b15', 'request_id': 'req-93641cfe-606e-4147-875e-1b5b563f83aa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.233793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1116efd72f43d5ab05225c05323b15', 'request_id': 'req-93641cfe-606e-4147-875e-1b5b563f83aa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.256533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1116efd72f43d5ab05225c05323b15', 'request_id': 'req-93641cfe-606e-4147-875e-1b5b563f83aa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.302805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] The token cache is being invalidate because project 1a1116efd72f43d5ab05225c05323b15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:58.305061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.306234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.308507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 933/1866] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:58 2022] DELETE /identity/v3/projects/1a1116efd72f43d5ab05225c05323b15 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:58.312813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975ac3e-f628-4876-8341-61c27d5927c3 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:58.338501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975ac3e-f628-4876-8341-61c27d5927c3 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:58.469054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:58.470909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:58.472185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:58.472675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] PATH_INFO: `/v3/users/dbcfe4ec08f14e53ae7b8a2fdb0eef32` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:58.474674 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbcfe4ec08f14e53ae7b8a2fdb0eef32)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:58.478382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:58.582816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbcfe4ec08f14e53ae7b8a2fdb0eef32'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.596786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbcfe4ec08f14e53ae7b8a2fdb0eef32'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.597536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbcfe4ec08f14e53ae7b8a2fdb0eef32'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.604859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbcfe4ec08f14e53ae7b8a2fdb0eef32'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.605565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cb0094a8-6ce1-4678-8bc8-fdb1ab970ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbcfe4ec08f14e53ae7b8a2fdb0eef32'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:58.621272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 934/1867] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:58 2022] DELETE /identity/v3/users/dbcfe4ec08f14e53ae7b8a2fdb0eef32 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:58.950339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:58.952027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:58.967561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93641cfe-606e-4147-875e-1b5b563f83aa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.041716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:59.056029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.057469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.057826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.058172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] PATH_INFO: `/v3/projects/d352c2b703c546f9a67a644c79b10a09` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.060456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d352c2b703c546f9a67a644c79b10a09)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.064008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.129430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd352c2b703c546f9a67a644c79b10a09', 'request_id': 'req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.142746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd352c2b703c546f9a67a644c79b10a09', 'request_id': 'req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.158458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd352c2b703c546f9a67a644c79b10a09', 'request_id': 'req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.174460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] The token cache is being invalidate because project d352c2b703c546f9a67a644c79b10a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:04:59.174788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.175653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfa63454-07e5-4bcc-ae1f-b006b4ec6321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.177898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 934/1868] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:04:58 2022] DELETE /identity/v3/projects/d352c2b703c546f9a67a644c79b10a09 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:59.182594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71744b61-5440-4591-9148-81b373154f5f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.182864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71744b61-5440-4591-9148-81b373154f5f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.183012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71744b61-5440-4591-9148-81b373154f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.205315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-71744b61-5440-4591-9148-81b373154f5f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:59.206944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-71744b61-5440-4591-9148-81b373154f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:59.212269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:59.325642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 935/1869] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:04:59.327257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788c4f4-4f3b-45ac-948c-83d0cc525295 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.327676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788c4f4-4f3b-45ac-948c-83d0cc525295 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.327829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788c4f4-4f3b-45ac-948c-83d0cc525295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.356428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0788c4f4-4f3b-45ac-948c-83d0cc525295 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:59.357902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0788c4f4-4f3b-45ac-948c-83d0cc525295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:59.369061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 935/1870] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:59.373400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71744b61-5440-4591-9148-81b373154f5f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.388143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71744b61-5440-4591-9148-81b373154f5f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.393720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.394409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.394697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.404380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.405620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:04:59.406155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.410105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.432238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 936/1871] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:04:59.434706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788c4f4-4f3b-45ac-948c-83d0cc525295 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.445645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788c4f4-4f3b-45ac-948c-83d0cc525295 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.450805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeda114-664e-420b-9e50-e8782fb28b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.451655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeda114-664e-420b-9e50-e8782fb28b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.452067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeda114-664e-420b-9e50-e8782fb28b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.452562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeda114-664e-420b-9e50-e8782fb28b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.453380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeda114-664e-420b-9e50-e8782fb28b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.455982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeda114-664e-420b-9e50-e8782fb28b07 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.485630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 936/1872] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:59 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:59.496339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.511928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d0676-9073-40e2-8447-d6cee31e8417 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.614097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.614783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.615072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.615350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.616094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.618049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.645645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 937/1873] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:59 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:59.646803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeda114-664e-420b-9e50-e8782fb28b07 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.655100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.657326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:59.660001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeda114-664e-420b-9e50-e8782fb28b07 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.665307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb3cd9-b3ab-41f1-90f1-58df718a9fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.666836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb3cd9-b3ab-41f1-90f1-58df718a9fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.667228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb3cd9-b3ab-41f1-90f1-58df718a9fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.667726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb3cd9-b3ab-41f1-90f1-58df718a9fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.668627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb3cd9-b3ab-41f1-90f1-58df718a9fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.670332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb3cd9-b3ab-41f1-90f1-58df718a9fb0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.670947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106f3e1-2b97-4e37-946e-743f04a45bde admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.676278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.677412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.677696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.677973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] PATH_INFO: `/v3/users/2bf3bc6ce197486eb02bf50eee7a148b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.679187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf3bc6ce197486eb02bf50eee7a148b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.681158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.693152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 937/1874] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:59.697290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb3cd9-b3ab-41f1-90f1-58df718a9fb0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.708001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb3cd9-b3ab-41f1-90f1-58df718a9fb0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.712801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cac2d-1ddb-4055-98a4-6320ab95011f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.713476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cac2d-1ddb-4055-98a4-6320ab95011f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.713759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cac2d-1ddb-4055-98a4-6320ab95011f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.714041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cac2d-1ddb-4055-98a4-6320ab95011f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.714777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cac2d-1ddb-4055-98a4-6320ab95011f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.716449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cac2d-1ddb-4055-98a4-6320ab95011f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.728499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 938/1875] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:04:59.732560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cac2d-1ddb-4055-98a4-6320ab95011f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.740917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf3bc6ce197486eb02bf50eee7a148b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.743062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cac2d-1ddb-4055-98a4-6320ab95011f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.747859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf3e69-fc22-40f7-a6b1-c9e29df99d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.748771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3e69-fc22-40f7-a6b1-c9e29df99d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.749057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3e69-fc22-40f7-a6b1-c9e29df99d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.749333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf3e69-fc22-40f7-a6b1-c9e29df99d5f admin admin] PATH_INFO: `/v3/projects/c9c43f5bdf9e412eae3e16b3c5f51782/users/48aca1461fab41b8b4b7a428a422279c/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.753716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf3bc6ce197486eb02bf50eee7a148b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.754411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf3bc6ce197486eb02bf50eee7a148b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.761976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf3bc6ce197486eb02bf50eee7a148b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.762733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-fe929a12-e87f-42b1-8b46-dd7b7a7e5c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf3bc6ce197486eb02bf50eee7a148b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:04:59.765892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf3e69-fc22-40f7-a6b1-c9e29df99d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c43f5bdf9e412eae3e16b3c5f51782, user_id=48aca1461fab41b8b4b7a428a422279c, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.769430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf3e69-fc22-40f7-a6b1-c9e29df99d5f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.773882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:59.779960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 938/1876] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:04:59 2022] DELETE /identity/v3/users/2bf3bc6ce197486eb02bf50eee7a148b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:04:59.784651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b619604-6b75-42d5-8223-05cd759b1b25 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.785398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b619604-6b75-42d5-8223-05cd759b1b25 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.785611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 939/1877] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:04:59 2022] PUT /identity/v3/projects/c9c43f5bdf9e412eae3e16b3c5f51782/users/48aca1461fab41b8b4b7a428a422279c/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:04:59.786051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b619604-6b75-42d5-8223-05cd759b1b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.796090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4cbd-e366-4e15-805d-2ac34ad5e36e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.796552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4cbd-e366-4e15-805d-2ac34ad5e36e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.796844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4cbd-e366-4e15-805d-2ac34ad5e36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.806075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5b619604-6b75-42d5-8223-05cd759b1b25 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:59.807296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5b619604-6b75-42d5-8223-05cd759b1b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:59.824657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-46dd4cbd-e366-4e15-805d-2ac34ad5e36e None None] MFA Rules not processed for user `48aca1461fab41b8b4b7a428a422279c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:04:59.825801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-46dd4cbd-e366-4e15-805d-2ac34ad5e36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:59.871428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 939/1878] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:59.885364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b619604-6b75-42d5-8223-05cd759b1b25 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.886787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5b619604-6b75-42d5-8223-05cd759b1b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:59.898871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:04:59.906226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 940/1879] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:04:59.912079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd4cbd-e366-4e15-805d-2ac34ad5e36e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:04:59.914630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-46dd4cbd-e366-4e15-805d-2ac34ad5e36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:04:59.931387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd4cbd-e366-4e15-805d-2ac34ad5e36e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.937223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd85f50-6ec9-41ff-917f-66d40429a021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.937894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85f50-6ec9-41ff-917f-66d40429a021 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.938213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85f50-6ec9-41ff-917f-66d40429a021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.938498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85f50-6ec9-41ff-917f-66d40429a021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.939186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd85f50-6ec9-41ff-917f-66d40429a021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.941154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd85f50-6ec9-41ff-917f-66d40429a021 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.956953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b619604-6b75-42d5-8223-05cd759b1b25 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:04:59.960731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efafc5-249b-4f6e-8119-5177d4d6f4dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:04:59.961392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efafc5-249b-4f6e-8119-5177d4d6f4dc service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:04:59.961704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efafc5-249b-4f6e-8119-5177d4d6f4dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:04:59.961964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efafc5-249b-4f6e-8119-5177d4d6f4dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:04:59.977319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efafc5-249b-4f6e-8119-5177d4d6f4dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:04:59.978858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efafc5-249b-4f6e-8119-5177d4d6f4dc service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:04:59.984340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 941/1880] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:04:59 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:04:59.988718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd85f50-6ec9-41ff-917f-66d40429a021 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:00.004374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 940/1881] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:00.119444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd85f50-6ec9-41ff-917f-66d40429a021 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:00.125572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541722-bf53-41be-b1da-e31460fee002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:00.127245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541722-bf53-41be-b1da-e31460fee002 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.127649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541722-bf53-41be-b1da-e31460fee002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.128011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd541722-bf53-41be-b1da-e31460fee002 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.128861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541722-bf53-41be-b1da-e31460fee002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:00.131080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd541722-bf53-41be-b1da-e31460fee002 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:00.163264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 942/1882] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:04:59 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:00.167823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efafc5-249b-4f6e-8119-5177d4d6f4dc service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:00.178880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efafc5-249b-4f6e-8119-5177d4d6f4dc service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:00.184065 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5febd-221d-4199-8b47-bae6ff1909cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:00.184893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5febd-221d-4199-8b47-bae6ff1909cc admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.185250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5febd-221d-4199-8b47-bae6ff1909cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.185635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5febd-221d-4199-8b47-bae6ff1909cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.187121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5febd-221d-4199-8b47-bae6ff1909cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:00.191239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5febd-221d-4199-8b47-bae6ff1909cc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:00.213330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 941/1883] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:00.214600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541722-bf53-41be-b1da-e31460fee002 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:00.234337 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd541722-bf53-41be-b1da-e31460fee002 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:00.239951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d455c7a-c6ec-4503-a0d6-348eae1a73e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:00.240875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d455c7a-c6ec-4503-a0d6-348eae1a73e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.241155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d455c7a-c6ec-4503-a0d6-348eae1a73e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.241437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d455c7a-c6ec-4503-a0d6-348eae1a73e9 admin admin] PATH_INFO: `/v3/projects/15368171f38647399b3be04008d3a0e4/users/eca4e990f0164d419744d4ae9ee84f33/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.263623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d455c7a-c6ec-4503-a0d6-348eae1a73e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15368171f38647399b3be04008d3a0e4, user_id=eca4e990f0164d419744d4ae9ee84f33, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:00.267208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d455c7a-c6ec-4503-a0d6-348eae1a73e9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:00.286156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 943/1884] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:00 2022] PUT /identity/v3/projects/15368171f38647399b3be04008d3a0e4/users/eca4e990f0164d419744d4ae9ee84f33/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:00.290703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5febd-221d-4199-8b47-bae6ff1909cc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:00.310749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5febd-221d-4199-8b47-bae6ff1909cc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:00.397817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:00.398679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.399072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.399710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.400738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:00.402661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:00.415997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 942/1885] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:00.421600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d455c7a-c6ec-4503-a0d6-348eae1a73e9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:00.434989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d455c7a-c6ec-4503-a0d6-348eae1a73e9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:00.442788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c11368-2a1d-4550-bddb-d32425892284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:00.443857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c11368-2a1d-4550-bddb-d32425892284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.444148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c11368-2a1d-4550-bddb-d32425892284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.444458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c11368-2a1d-4550-bddb-d32425892284 admin admin] PATH_INFO: `/v3/projects/15368171f38647399b3be04008d3a0e4/users/eca4e990f0164d419744d4ae9ee84f33/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.448929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:00.449365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c11368-2a1d-4550-bddb-d32425892284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15368171f38647399b3be04008d3a0e4, user_id=eca4e990f0164d419744d4ae9ee84f33, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:00.451319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:00.452407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c11368-2a1d-4550-bddb-d32425892284 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:00.466130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 944/1886] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:00 2022] PUT /identity/v3/projects/15368171f38647399b3be04008d3a0e4/users/eca4e990f0164d419744d4ae9ee84f33/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:00.476020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59062637-2f7f-46c7-8cd3-0bb4bc89b07a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:00.478119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d427e2-4d2a-4c9a-bf9a-289add7f7cb5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.478727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d427e2-4d2a-4c9a-bf9a-289add7f7cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.479165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d427e2-4d2a-4c9a-bf9a-289add7f7cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.514302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:00.545946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-50d427e2-4d2a-4c9a-bf9a-289add7f7cb5 None None] MFA Rules not processed for user `eca4e990f0164d419744d4ae9ee84f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:00.547231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-50d427e2-4d2a-4c9a-bf9a-289add7f7cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:00.570821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:00.572585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.573009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.573433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] PATH_INFO: `/v3/projects/2d37e17fe95949f291de5394c3697ae9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.575697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d37e17fe95949f291de5394c3697ae9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:00.579509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:00.715704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d37e17fe95949f291de5394c3697ae9', 'request_id': 'req-a371d651-44ec-4a65-920d-d4e5bcd0732e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:00.718473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 945/1887] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:00.730423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d427e2-4d2a-4c9a-bf9a-289add7f7cb5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:00.732186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-50d427e2-4d2a-4c9a-bf9a-289add7f7cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:00.738191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d37e17fe95949f291de5394c3697ae9', 'request_id': 'req-a371d651-44ec-4a65-920d-d4e5bcd0732e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:00.749370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d37e17fe95949f291de5394c3697ae9', 'request_id': 'req-a371d651-44ec-4a65-920d-d4e5bcd0732e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:00.767897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:00.772818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] The token cache is being invalidate because project 2d37e17fe95949f291de5394c3697ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:00.774619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:00.775630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:00.781647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 943/1888] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:00 2022] DELETE /identity/v3/projects/2d37e17fe95949f291de5394c3697ae9 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:00.784808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:00.787856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:00.871061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d427e2-4d2a-4c9a-bf9a-289add7f7cb5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:00.903985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d651-44ec-4a65-920d-d4e5bcd0732e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:00.909928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:00.910822 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.911136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.911389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] PATH_INFO: `/v3/users/0e7001445b8c4cd39c066f902117c4e9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.912653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e7001445b8c4cd39c066f902117c4e9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:00.914635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:00.942512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867ab64-91e1-46fb-92b7-c752c6f90da7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:00.942890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867ab64-91e1-46fb-92b7-c752c6f90da7 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:00.943194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867ab64-91e1-46fb-92b7-c752c6f90da7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:00.943465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867ab64-91e1-46fb-92b7-c752c6f90da7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:00.945827 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9867ab64-91e1-46fb-92b7-c752c6f90da7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:00.975171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e7001445b8c4cd39c066f902117c4e9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:00.994238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:01.011177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e7001445b8c4cd39c066f902117c4e9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.012598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e7001445b8c4cd39c066f902117c4e9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.023354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e7001445b8c4cd39c066f902117c4e9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.024460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e7001445b8c4cd39c066f902117c4e9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.041563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867ab64-91e1-46fb-92b7-c752c6f90da7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:01.043074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867ab64-91e1-46fb-92b7-c752c6f90da7 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:01.047282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 944/1889] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:00 2022] DELETE /identity/v3/users/0e7001445b8c4cd39c066f902117c4e9 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:01.060407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 946/1890] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:05:01.691982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:01.693557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:01.826016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d09741-e90b-4eb8-abf8-f57bf6e83a13 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:01.846826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:01.849292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:01.849598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:01.849886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] PATH_INFO: `/v3/projects/b035cbab320747f79860603860f12459` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:01.851373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b035cbab320747f79860603860f12459)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:01.860972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:01.951925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b035cbab320747f79860603860f12459', 'request_id': 'req-949af174-7288-4e40-80b0-709f4bf976a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.960614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b035cbab320747f79860603860f12459', 'request_id': 'req-949af174-7288-4e40-80b0-709f4bf976a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.968614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b035cbab320747f79860603860f12459', 'request_id': 'req-949af174-7288-4e40-80b0-709f4bf976a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.993064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] The token cache is being invalidate because project b035cbab320747f79860603860f12459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:01.993064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.993064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-949af174-7288-4e40-80b0-709f4bf976a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:01.993064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 945/1891] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:01 2022] DELETE /identity/v3/projects/b035cbab320747f79860603860f12459 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:01.995445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7738a8-5aae-4b37-9312-9ded4282b4b6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:01.996183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7738a8-5aae-4b37-9312-9ded4282b4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:01.996272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7738a8-5aae-4b37-9312-9ded4282b4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.017905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4e7738a8-5aae-4b37-9312-9ded4282b4b6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:02.019125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4e7738a8-5aae-4b37-9312-9ded4282b4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:02.100344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 947/1892] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:02.110643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f47f-e6c5-4d7a-8247-9756d2b7cd05 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.111156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f47f-e6c5-4d7a-8247-9756d2b7cd05 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.111447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f47f-e6c5-4d7a-8247-9756d2b7cd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.136951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-84f6f47f-e6c5-4d7a-8247-9756d2b7cd05 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:02.138371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-84f6f47f-e6c5-4d7a-8247-9756d2b7cd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:02.152498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 946/1893] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:02.153513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7738a8-5aae-4b37-9312-9ded4282b4b6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:02.165189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7738a8-5aae-4b37-9312-9ded4282b4b6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:02.170234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:02.170954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.171317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.171735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.172950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:02.173353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:02.175524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:02.190569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 948/1894] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 12:05:02.194858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6f47f-e6c5-4d7a-8247-9756d2b7cd05 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:02.206389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6f47f-e6c5-4d7a-8247-9756d2b7cd05 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:02.211585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdeee2-13db-4be6-8cf2-8db4970d54d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:02.211828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdeee2-13db-4be6-8cf2-8db4970d54d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.212282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdeee2-13db-4be6-8cf2-8db4970d54d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.212642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdeee2-13db-4be6-8cf2-8db4970d54d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.213400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdeee2-13db-4be6-8cf2-8db4970d54d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:02.215428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdeee2-13db-4be6-8cf2-8db4970d54d8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:02.248224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 947/1895] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:02 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:02.249854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:02.264635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30855a6-1612-46ae-9d36-52a2407576ce admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:02.278227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:02.341218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c5c6e-142d-4053-949c-0164fdc34d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:02.342078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c5c6e-142d-4053-949c-0164fdc34d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.342434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c5c6e-142d-4053-949c-0164fdc34d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.342723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c5c6e-142d-4053-949c-0164fdc34d03 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.343435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c5c6e-142d-4053-949c-0164fdc34d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:02.346304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c5c6e-142d-4053-949c-0164fdc34d03 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:02.374577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 949/1896] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:02 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:02.377046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdeee2-13db-4be6-8cf2-8db4970d54d8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:02.389004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdeee2-13db-4be6-8cf2-8db4970d54d8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:02.394151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827494-2dd3-4c9e-a2eb-d393b8fb2e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:02.395584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827494-2dd3-4c9e-a2eb-d393b8fb2e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.395852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827494-2dd3-4c9e-a2eb-d393b8fb2e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.396219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e827494-2dd3-4c9e-a2eb-d393b8fb2e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.397351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e827494-2dd3-4c9e-a2eb-d393b8fb2e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:02.399051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e827494-2dd3-4c9e-a2eb-d393b8fb2e40 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:02.420917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 948/1897] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:02.425284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c5c6e-142d-4053-949c-0164fdc34d03 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:02.442544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c5c6e-142d-4053-949c-0164fdc34d03 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:02.447888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9476d7-1de2-42f5-bdb7-a4e13a5e71d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:02.448917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9476d7-1de2-42f5-bdb7-a4e13a5e71d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.450474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9476d7-1de2-42f5-bdb7-a4e13a5e71d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.450860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9476d7-1de2-42f5-bdb7-a4e13a5e71d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.451654 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9476d7-1de2-42f5-bdb7-a4e13a5e71d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:02.453415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9476d7-1de2-42f5-bdb7-a4e13a5e71d6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:02.471889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 950/1898] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:02.475869 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827494-2dd3-4c9e-a2eb-d393b8fb2e40 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:02.487180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e827494-2dd3-4c9e-a2eb-d393b8fb2e40 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:02.492660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce709d4-11f7-4643-a6f0-30f2de2be04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:02.493579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce709d4-11f7-4643-a6f0-30f2de2be04f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.493866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce709d4-11f7-4643-a6f0-30f2de2be04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.494147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce709d4-11f7-4643-a6f0-30f2de2be04f admin admin] PATH_INFO: `/v3/projects/8db39bad02544170b9bc9fb25ec2c025/users/ac6de733a227484e8123ebc8b1e662d6/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.511510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce709d4-11f7-4643-a6f0-30f2de2be04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db39bad02544170b9bc9fb25ec2c025, user_id=ac6de733a227484e8123ebc8b1e662d6, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:02.514280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce709d4-11f7-4643-a6f0-30f2de2be04f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:02.531836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 949/1899] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:02 2022] PUT /identity/v3/projects/8db39bad02544170b9bc9fb25ec2c025/users/ac6de733a227484e8123ebc8b1e662d6/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:02.544067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a908eac-6b71-4ad3-8f56-685505ae5e2e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.544798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a908eac-6b71-4ad3-8f56-685505ae5e2e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.546002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a908eac-6b71-4ad3-8f56-685505ae5e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.584603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0a908eac-6b71-4ad3-8f56-685505ae5e2e None None] MFA Rules not processed for user `ac6de733a227484e8123ebc8b1e662d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:02.585848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0a908eac-6b71-4ad3-8f56-685505ae5e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:02.690143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 951/1900] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:02.694257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce709d4-11f7-4643-a6f0-30f2de2be04f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:02.699327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:02.772323 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce709d4-11f7-4643-a6f0-30f2de2be04f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:02.779155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7128d8-a716-4590-ad06-aa560da9e099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:02.779990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7128d8-a716-4590-ad06-aa560da9e099 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.780974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7128d8-a716-4590-ad06-aa560da9e099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.781379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7128d8-a716-4590-ad06-aa560da9e099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.783071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7128d8-a716-4590-ad06-aa560da9e099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:02.785512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7128d8-a716-4590-ad06-aa560da9e099 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:02.835151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 950/1901] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:02 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:02.837109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a908eac-6b71-4ad3-8f56-685505ae5e2e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:02.854480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a908eac-6b71-4ad3-8f56-685505ae5e2e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:02.955450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b7346-0f3c-4202-80ff-92d2ae71167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:02.956392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b7346-0f3c-4202-80ff-92d2ae71167f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:02.956749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b7346-0f3c-4202-80ff-92d2ae71167f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:02.957089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b7346-0f3c-4202-80ff-92d2ae71167f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:02.958029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b7346-0f3c-4202-80ff-92d2ae71167f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:02.960241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b7346-0f3c-4202-80ff-92d2ae71167f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:02.995809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 952/1902] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:02 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:02.996669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7128d8-a716-4590-ad06-aa560da9e099 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:03.009969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7128d8-a716-4590-ad06-aa560da9e099 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:03.018024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7f574-6daa-466c-b33b-44f9f94226e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:03.020136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7f574-6daa-466c-b33b-44f9f94226e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:03.020517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7f574-6daa-466c-b33b-44f9f94226e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:03.020870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7f574-6daa-466c-b33b-44f9f94226e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:03.029382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7f574-6daa-466c-b33b-44f9f94226e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:03.035864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7f574-6daa-466c-b33b-44f9f94226e1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:03.053205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 951/1903] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:03.054428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b7346-0f3c-4202-80ff-92d2ae71167f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:03.065983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b7346-0f3c-4202-80ff-92d2ae71167f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:03.071383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306fab5-c106-43b0-9d87-7539037d70da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:03.072445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306fab5-c106-43b0-9d87-7539037d70da admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:03.072618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306fab5-c106-43b0-9d87-7539037d70da admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:03.072906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306fab5-c106-43b0-9d87-7539037d70da admin admin] PATH_INFO: `/v3/projects/d2e39e6e9b9a473fb655d44d68ab4d7d/users/25aca5ab79cc4dee86c98bceca8ce9d8/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:03.076724 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:03.093928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306fab5-c106-43b0-9d87-7539037d70da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e39e6e9b9a473fb655d44d68ab4d7d, user_id=25aca5ab79cc4dee86c98bceca8ce9d8, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:03.098500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306fab5-c106-43b0-9d87-7539037d70da admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:03.118439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 953/1904] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:03 2022] PUT /identity/v3/projects/d2e39e6e9b9a473fb655d44d68ab4d7d/users/25aca5ab79cc4dee86c98bceca8ce9d8/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:03.122729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7f574-6daa-466c-b33b-44f9f94226e1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:03.137430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7f574-6daa-466c-b33b-44f9f94226e1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:03.280868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526f299-8b92-43a7-bf1a-b13a8a02c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:03.281721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526f299-8b92-43a7-bf1a-b13a8a02c52c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:03.282160 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526f299-8b92-43a7-bf1a-b13a8a02c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:03.282522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526f299-8b92-43a7-bf1a-b13a8a02c52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:03.283642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526f299-8b92-43a7-bf1a-b13a8a02c52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:03.288793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526f299-8b92-43a7-bf1a-b13a8a02c52c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:03.316913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 952/1905] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:03.320889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306fab5-c106-43b0-9d87-7539037d70da admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:03.339483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306fab5-c106-43b0-9d87-7539037d70da admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:03.348713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:03.349995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:03.351103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:03.351634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] PATH_INFO: `/v3/projects/d2e39e6e9b9a473fb655d44d68ab4d7d/users/25aca5ab79cc4dee86c98bceca8ce9d8/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:03.354380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e39e6e9b9a473fb655d44d68ab4d7d, user_id=25aca5ab79cc4dee86c98bceca8ce9d8, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:03.358294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:03.375431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 954/1906] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:03 2022] PUT /identity/v3/projects/d2e39e6e9b9a473fb655d44d68ab4d7d/users/25aca5ab79cc4dee86c98bceca8ce9d8/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:03.385940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83ca4d-09b7-4a55-94f6-8302824ae9d3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:03.386557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83ca4d-09b7-4a55-94f6-8302824ae9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:03.386924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83ca4d-09b7-4a55-94f6-8302824ae9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:03.428840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4c83ca4d-09b7-4a55-94f6-8302824ae9d3 None None] MFA Rules not processed for user `25aca5ab79cc4dee86c98bceca8ce9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:03.430784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4c83ca4d-09b7-4a55-94f6-8302824ae9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:03.621649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:03.630508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 953/1907] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:03.645526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:03.647695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:03.719616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea1266-087a-48c3-a5f5-f726fc452e43 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:03.723416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065272c-8291-4563-8100-ff6af5e8ae2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:03.724524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065272c-8291-4563-8100-ff6af5e8ae2c service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:03.724524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065272c-8291-4563-8100-ff6af5e8ae2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:03.724761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065272c-8291-4563-8100-ff6af5e8ae2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:03.736181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065272c-8291-4563-8100-ff6af5e8ae2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:03.737649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065272c-8291-4563-8100-ff6af5e8ae2c service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:03.752322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 955/1908] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:04.293284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83ca4d-09b7-4a55-94f6-8302824ae9d3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:04.303803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83ca4d-09b7-4a55-94f6-8302824ae9d3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:04.310504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25aca5ab79cc4dee86c98bceca8ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2e39e6e9b9a473fb655d44d68ab4d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:04.311538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:04.312044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:04.312603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:04.313658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:04.314198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:04.316043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:04.352698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 954/1909] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:04.352927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065272c-8291-4563-8100-ff6af5e8ae2c service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:04.361041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065272c-8291-4563-8100-ff6af5e8ae2c service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:04.365959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b76925-37d2-42bf-9b3d-5bd49ca3b4ec tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25aca5ab79cc4dee86c98bceca8ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2e39e6e9b9a473fb655d44d68ab4d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:04.366650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b76925-37d2-42bf-9b3d-5bd49ca3b4ec tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:04.366947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b76925-37d2-42bf-9b3d-5bd49ca3b4ec tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:04.367235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b76925-37d2-42bf-9b3d-5bd49ca3b4ec tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:04.367946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b76925-37d2-42bf-9b3d-5bd49ca3b4ec tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:04.369944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b76925-37d2-42bf-9b3d-5bd49ca3b4ec tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:04.411808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 956/1910] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:04 2022] POST /identity/v3/projects => generated 355 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:04.415671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:04.434107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e87e7d-da5d-4c09-9970-159243ff258b tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:04.569375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25aca5ab79cc4dee86c98bceca8ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2e39e6e9b9a473fb655d44d68ab4d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:04.571992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:04.574623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:04.574956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:04.575848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:04.576351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:04.582262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:04.622267 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 955/1911] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:04.704620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b76925-37d2-42bf-9b3d-5bd49ca3b4ec tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:04.721658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b76925-37d2-42bf-9b3d-5bd49ca3b4ec tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:04.751102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25aca5ab79cc4dee86c98bceca8ce9d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2e39e6e9b9a473fb655d44d68ab4d7d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:04.752718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:04.753008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:04.753294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] PATH_INFO: `/v3/projects/4eaae7ea47244a41b02e97f735fa6b4e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:04.755731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4eaae7ea47244a41b02e97f735fa6b4e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:04.762260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:04.870584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eaae7ea47244a41b02e97f735fa6b4e', 'request_id': 'req-77705bc3-c9e7-4795-90e8-255b307b2d01'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:04.881059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eaae7ea47244a41b02e97f735fa6b4e', 'request_id': 'req-77705bc3-c9e7-4795-90e8-255b307b2d01'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:04.890672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eaae7ea47244a41b02e97f735fa6b4e', 'request_id': 'req-77705bc3-c9e7-4795-90e8-255b307b2d01'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:04.908640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] The token cache is being invalidate because project 4eaae7ea47244a41b02e97f735fa6b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:04.908949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:04.909948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77705bc3-c9e7-4795-90e8-255b307b2d01 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:04.913505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 957/1912] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:04 2022] DELETE /identity/v3/projects/4eaae7ea47244a41b02e97f735fa6b4e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:05.404419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:05.407865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:05.522793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5972555-8b22-4be6-9a28-3b72e8907e20 tempest-VolumeQuotaClassesTest-1866612496 tempest-VolumeQuotaClassesTest-1866612496-project-admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:05.620908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:05.621827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:05.622111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:05.622391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] PATH_INFO: `/v3/users/ac6de733a227484e8123ebc8b1e662d6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:05.623680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac6de733a227484e8123ebc8b1e662d6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:05.626511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:05.694205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:05.722318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac6de733a227484e8123ebc8b1e662d6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:05.742468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac6de733a227484e8123ebc8b1e662d6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:05.743368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac6de733a227484e8123ebc8b1e662d6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:05.753986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac6de733a227484e8123ebc8b1e662d6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:05.754532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac6de733a227484e8123ebc8b1e662d6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:05.778088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 956/1913] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:05 2022] DELETE /identity/v3/users/ac6de733a227484e8123ebc8b1e662d6 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:05.779331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b07ba-e16d-4bf1-8dac-fab23f63b239 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:05.779798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b07ba-e16d-4bf1-8dac-fab23f63b239 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:05.779945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b07ba-e16d-4bf1-8dac-fab23f63b239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:05.805215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b13b07ba-e16d-4bf1-8dac-fab23f63b239 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:05.807112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b13b07ba-e16d-4bf1-8dac-fab23f63b239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:05.849297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:05.917984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 958/1914] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:05.936647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:05.941792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:06.034314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748349e-a709-4720-893e-8dea2ebc99a2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:06.040962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:06.042259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:06.042783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:06.043165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:06.058671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:06.060241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:06.072856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 957/1915] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:06.510469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b07ba-e16d-4bf1-8dac-fab23f63b239 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:06.512295 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b13b07ba-e16d-4bf1-8dac-fab23f63b239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:06.534258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:06.536464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:06.562553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a8422-6c81-410b-b08d-531b5f170269 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:06.632733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:06.633655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:06.633942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:06.634340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] PATH_INFO: `/v3/projects/8db39bad02544170b9bc9fb25ec2c025` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:06.640626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db39bad02544170b9bc9fb25ec2c025)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:06.644420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b07ba-e16d-4bf1-8dac-fab23f63b239 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:06.648970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:06.649857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:06.650704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:06.651067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:06.651597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:06.654151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:06.755090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db39bad02544170b9bc9fb25ec2c025', 'request_id': 'req-76a8e19f-2194-4b9a-ae2d-2821e85119f5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:06.764948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db39bad02544170b9bc9fb25ec2c025', 'request_id': 'req-76a8e19f-2194-4b9a-ae2d-2821e85119f5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:06.775110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db39bad02544170b9bc9fb25ec2c025', 'request_id': 'req-76a8e19f-2194-4b9a-ae2d-2821e85119f5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:06.810931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] The token cache is being invalidate because project 8db39bad02544170b9bc9fb25ec2c025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:06.811544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:06.812432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:06.815038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 958/1916] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:06 2022] DELETE /identity/v3/projects/8db39bad02544170b9bc9fb25ec2c025 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:06.819273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:06.821481 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:06.824235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:06.825755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:06.860785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 959/1917] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:06.875270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:06.932090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8e19f-2194-4b9a-ae2d-2821e85119f5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:06.938282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:06.941891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:06.942456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:06.943064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] PATH_INFO: `/v3/users/25aca5ab79cc4dee86c98bceca8ce9d8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:06.944599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25aca5ab79cc4dee86c98bceca8ce9d8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:06.948342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:07.046874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25aca5ab79cc4dee86c98bceca8ce9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.066757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25aca5ab79cc4dee86c98bceca8ce9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.067702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25aca5ab79cc4dee86c98bceca8ce9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.085468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25aca5ab79cc4dee86c98bceca8ce9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.086007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74726d2b-3a0b-481d-b21d-fec49207dfd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25aca5ab79cc4dee86c98bceca8ce9d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.116810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 959/1918] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:06 2022] DELETE /identity/v3/users/25aca5ab79cc4dee86c98bceca8ce9d8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:07.619046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:07.660784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24e0bf-e55c-488e-809a-b1b90914bb5b service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:07.748067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:07.749022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:07.749310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:07.749586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] PATH_INFO: `/v3/projects/d2e39e6e9b9a473fb655d44d68ab4d7d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:07.751873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e39e6e9b9a473fb655d44d68ab4d7d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:07.754682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:07.794545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:07.821573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e39e6e9b9a473fb655d44d68ab4d7d', 'request_id': 'req-6409a641-1be1-4a05-bb2b-4cbef2acc382'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.831594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e39e6e9b9a473fb655d44d68ab4d7d', 'request_id': 'req-6409a641-1be1-4a05-bb2b-4cbef2acc382'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.839148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e39e6e9b9a473fb655d44d68ab4d7d', 'request_id': 'req-6409a641-1be1-4a05-bb2b-4cbef2acc382'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.851202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] The token cache is being invalidate because project d2e39e6e9b9a473fb655d44d68ab4d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:07.851488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.852153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6409a641-1be1-4a05-bb2b-4cbef2acc382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:07.853737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 960/1919] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:07 2022] DELETE /identity/v3/projects/d2e39e6e9b9a473fb655d44d68ab4d7d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:07.858795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dc09f-fbb6-4875-99a3-8703ee2ef100 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:07.859266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dc09f-fbb6-4875-99a3-8703ee2ef100 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:07.859425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dc09f-fbb6-4875-99a3-8703ee2ef100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:07.878424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0b6dc09f-fbb6-4875-99a3-8703ee2ef100 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:07.879569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b6dc09f-fbb6-4875-99a3-8703ee2ef100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:07.947751 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 960/1920] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:07.956327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010dbd5-b564-4673-8da5-f85c332a21db None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:07.956621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010dbd5-b564-4673-8da5-f85c332a21db None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:07.956908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010dbd5-b564-4673-8da5-f85c332a21db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:07.975941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3010dbd5-b564-4673-8da5-f85c332a21db None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:07.977219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3010dbd5-b564-4673-8da5-f85c332a21db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:07.985665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 961/1921] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:07.990685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dc09f-fbb6-4875-99a3-8703ee2ef100 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.001517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dc09f-fbb6-4875-99a3-8703ee2ef100 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.006164 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.006837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.007131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.007406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.008141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:08.008565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.010177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.026611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 961/1922] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:08.027724 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010dbd5-b564-4673-8da5-f85c332a21db None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.038662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010dbd5-b564-4673-8da5-f85c332a21db None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.044532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ba8fd-6548-47f3-9df5-56b31939ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.045304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba8fd-6548-47f3-9df5-56b31939ad4c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.045596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba8fd-6548-47f3-9df5-56b31939ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.046318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba8fd-6548-47f3-9df5-56b31939ad4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.047613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ba8fd-6548-47f3-9df5-56b31939ad4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.049770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ba8fd-6548-47f3-9df5-56b31939ad4c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.079955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 962/1923] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:08 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:08.084179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.095658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08ece2-113e-4bbc-805a-9ccc44950c55 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.131372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:08.158890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574e3db-b1e5-4f11-8c7e-577c59d171b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.159579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e3db-b1e5-4f11-8c7e-577c59d171b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.159864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e3db-b1e5-4f11-8c7e-577c59d171b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.160143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e3db-b1e5-4f11-8c7e-577c59d171b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.160890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574e3db-b1e5-4f11-8c7e-577c59d171b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.162829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574e3db-b1e5-4f11-8c7e-577c59d171b7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.185344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 962/1924] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:08 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:08.189719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ba8fd-6548-47f3-9df5-56b31939ad4c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.200715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ba8fd-6548-47f3-9df5-56b31939ad4c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.206050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f9e20-2886-4e29-b9e2-fc4acb73efef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.206717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9e20-2886-4e29-b9e2-fc4acb73efef admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.207007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9e20-2886-4e29-b9e2-fc4acb73efef admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.207290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f9e20-2886-4e29-b9e2-fc4acb73efef admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.208029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f9e20-2886-4e29-b9e2-fc4acb73efef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.209684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f9e20-2886-4e29-b9e2-fc4acb73efef admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.221679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 963/1925] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:08.227258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574e3db-b1e5-4f11-8c7e-577c59d171b7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.238647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574e3db-b1e5-4f11-8c7e-577c59d171b7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.243378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1561c-0f0d-4378-b2ee-35f88c19b059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.244060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1561c-0f0d-4378-b2ee-35f88c19b059 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.244374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1561c-0f0d-4378-b2ee-35f88c19b059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.244657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1561c-0f0d-4378-b2ee-35f88c19b059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.245382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1561c-0f0d-4378-b2ee-35f88c19b059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.246998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1561c-0f0d-4378-b2ee-35f88c19b059 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.257569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 963/1926] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:08.261815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f9e20-2886-4e29-b9e2-fc4acb73efef admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.272776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f9e20-2886-4e29-b9e2-fc4acb73efef admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.277325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514dad0-5306-468f-892a-22a93d5b425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.278202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514dad0-5306-468f-892a-22a93d5b425a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.278513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514dad0-5306-468f-892a-22a93d5b425a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.278792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514dad0-5306-468f-892a-22a93d5b425a admin admin] PATH_INFO: `/v3/projects/454f733e042b460e9db049dff931b839/users/b8b45ca6d5cd4cc4b7ef359845e9f7de/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.294307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514dad0-5306-468f-892a-22a93d5b425a admin admin] RBAC: Authorizing `identity:create_grant(project_id=454f733e042b460e9db049dff931b839, user_id=b8b45ca6d5cd4cc4b7ef359845e9f7de, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.297673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514dad0-5306-468f-892a-22a93d5b425a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.312627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 964/1927] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:08 2022] PUT /identity/v3/projects/454f733e042b460e9db049dff931b839/users/b8b45ca6d5cd4cc4b7ef359845e9f7de/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:08.324352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875eb551-a69f-4318-90cc-b6a0570b84f2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.324678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875eb551-a69f-4318-90cc-b6a0570b84f2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.324989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875eb551-a69f-4318-90cc-b6a0570b84f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.354993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-875eb551-a69f-4318-90cc-b6a0570b84f2 None None] MFA Rules not processed for user `b8b45ca6d5cd4cc4b7ef359845e9f7de`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:08.356236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-875eb551-a69f-4318-90cc-b6a0570b84f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:08.424302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 964/1928] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:08.428645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514dad0-5306-468f-892a-22a93d5b425a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.490828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514dad0-5306-468f-892a-22a93d5b425a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.495568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d20e9-5a28-4a44-84d8-d7c9ad5bb490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.496267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d20e9-5a28-4a44-84d8-d7c9ad5bb490 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.496555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d20e9-5a28-4a44-84d8-d7c9ad5bb490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.496837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d20e9-5a28-4a44-84d8-d7c9ad5bb490 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.497677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d20e9-5a28-4a44-84d8-d7c9ad5bb490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.499604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d20e9-5a28-4a44-84d8-d7c9ad5bb490 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.531413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 965/1929] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:08 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:08.537056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875eb551-a69f-4318-90cc-b6a0570b84f2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.556139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875eb551-a69f-4318-90cc-b6a0570b84f2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.642191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a51a3-2e23-46b5-b309-e6e121f99d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.642964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a51a3-2e23-46b5-b309-e6e121f99d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.643349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a51a3-2e23-46b5-b309-e6e121f99d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.643704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a51a3-2e23-46b5-b309-e6e121f99d03 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.644550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a51a3-2e23-46b5-b309-e6e121f99d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.647084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a51a3-2e23-46b5-b309-e6e121f99d03 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.673603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 965/1930] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:08 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:08.675251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d20e9-5a28-4a44-84d8-d7c9ad5bb490 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.686893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d20e9-5a28-4a44-84d8-d7c9ad5bb490 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.692676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef090f71-4ceb-434d-b646-e40e83c0290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.693357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef090f71-4ceb-434d-b646-e40e83c0290b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.693690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef090f71-4ceb-434d-b646-e40e83c0290b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.693970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef090f71-4ceb-434d-b646-e40e83c0290b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.694743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef090f71-4ceb-434d-b646-e40e83c0290b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.696512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef090f71-4ceb-434d-b646-e40e83c0290b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.710139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 966/1931] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:08.714354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a51a3-2e23-46b5-b309-e6e121f99d03 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.725157 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a51a3-2e23-46b5-b309-e6e121f99d03 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.731611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688450d4-8668-433d-9cca-df84b925d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.733025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688450d4-8668-433d-9cca-df84b925d52e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.733394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688450d4-8668-433d-9cca-df84b925d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.733737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688450d4-8668-433d-9cca-df84b925d52e admin admin] PATH_INFO: `/v3/projects/bf59d9654e9e4f5787da03c6a36a76c9/users/54ac2fdc92214636a8f238e1fee2b9a0/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.753039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688450d4-8668-433d-9cca-df84b925d52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf59d9654e9e4f5787da03c6a36a76c9, user_id=54ac2fdc92214636a8f238e1fee2b9a0, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.755958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688450d4-8668-433d-9cca-df84b925d52e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.774401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 966/1932] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:08 2022] PUT /identity/v3/projects/bf59d9654e9e4f5787da03c6a36a76c9/users/54ac2fdc92214636a8f238e1fee2b9a0/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:08.778801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef090f71-4ceb-434d-b646-e40e83c0290b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.790081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef090f71-4ceb-434d-b646-e40e83c0290b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.824497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:08.869978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74752757-44c2-43a3-8ae5-5f877dd81ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.870740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752757-44c2-43a3-8ae5-5f877dd81ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.871018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752757-44c2-43a3-8ae5-5f877dd81ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.871488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752757-44c2-43a3-8ae5-5f877dd81ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.872265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74752757-44c2-43a3-8ae5-5f877dd81ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.876412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74752757-44c2-43a3-8ae5-5f877dd81ad7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.888286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 967/1933] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:08.892775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688450d4-8668-433d-9cca-df84b925d52e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:08.903240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688450d4-8668-433d-9cca-df84b925d52e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:08.912354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:08.912354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.912354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.912354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] PATH_INFO: `/v3/projects/bf59d9654e9e4f5787da03c6a36a76c9/users/54ac2fdc92214636a8f238e1fee2b9a0/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.912929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf59d9654e9e4f5787da03c6a36a76c9, user_id=54ac2fdc92214636a8f238e1fee2b9a0, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:08.915753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:08.921686 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:08.929319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 967/1934] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:08 2022] PUT /identity/v3/projects/bf59d9654e9e4f5787da03c6a36a76c9/users/54ac2fdc92214636a8f238e1fee2b9a0/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:08.939560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d30a-7029-46f1-a17d-f7f9a1d110ed None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:08.939868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d30a-7029-46f1-a17d-f7f9a1d110ed None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:08.940246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d30a-7029-46f1-a17d-f7f9a1d110ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:08.966429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7370d30a-7029-46f1-a17d-f7f9a1d110ed None None] MFA Rules not processed for user `54ac2fdc92214636a8f238e1fee2b9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:08.967632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7370d30a-7029-46f1-a17d-f7f9a1d110ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:09.038930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 968/1935] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:08 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:09.052470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:09.054591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:09.115645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf17775-435d-4542-97c1-2a914cfee369 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:09.119575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:09.120269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:09.120560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:09.120848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:09.130232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:09.131672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:09.141631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 968/1936] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:09.228177 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370d30a-7029-46f1-a17d-f7f9a1d110ed None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:09.239553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370d30a-7029-46f1-a17d-f7f9a1d110ed None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:09.243206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906ecaa-0ca9-4819-913a-e5b3c5d33851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:09.243859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ecaa-0ca9-4819-913a-e5b3c5d33851 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:09.244148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ecaa-0ca9-4819-913a-e5b3c5d33851 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:09.244459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ecaa-0ca9-4819-913a-e5b3c5d33851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:09.303562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906ecaa-0ca9-4819-913a-e5b3c5d33851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:09.305050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906ecaa-0ca9-4819-913a-e5b3c5d33851 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:09.315312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 969/1937] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:12.229473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:12.231081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:12.321412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df4242-e634-4acd-97ba-d47e05098757 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:12.326907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:12.327898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:12.328512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:12.328899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] PATH_INFO: `/v3/users/48aca1461fab41b8b4b7a428a422279c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:12.330265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48aca1461fab41b8b4b7a428a422279c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:12.332787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:12.395821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48aca1461fab41b8b4b7a428a422279c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:12.413069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48aca1461fab41b8b4b7a428a422279c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:12.413901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48aca1461fab41b8b4b7a428a422279c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:12.423993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48aca1461fab41b8b4b7a428a422279c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:12.425125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48aca1461fab41b8b4b7a428a422279c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:12.460364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 969/1938] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:12 2022] DELETE /identity/v3/users/48aca1461fab41b8b4b7a428a422279c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:12.462913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43265954-81ad-473d-9721-42cab01ebce2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:12.463389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43265954-81ad-473d-9721-42cab01ebce2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:12.463474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43265954-81ad-473d-9721-42cab01ebce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:12.481918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-43265954-81ad-473d-9721-42cab01ebce2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:12.483144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-43265954-81ad-473d-9721-42cab01ebce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:12.538500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:12.559079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 970/1939] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:12.569168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:12.570546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:12.597757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:12.645536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5309-d357-4c26-8f4b-0a229bc41c88 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:12.649396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866308c-f50c-45f6-ad5b-9680b79312cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:12.650055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866308c-f50c-45f6-ad5b-9680b79312cc service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:12.650341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866308c-f50c-45f6-ad5b-9680b79312cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:12.650621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866308c-f50c-45f6-ad5b-9680b79312cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:12.661096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866308c-f50c-45f6-ad5b-9680b79312cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:12.666250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866308c-f50c-45f6-ad5b-9680b79312cc service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:12.677085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 970/1940] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:05:13.044030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43265954-81ad-473d-9721-42cab01ebce2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:13.048605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-43265954-81ad-473d-9721-42cab01ebce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:13.073064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43265954-81ad-473d-9721-42cab01ebce2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:13.079047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:13.080051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:13.080381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:13.080664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] PATH_INFO: `/v3/projects/c9c43f5bdf9e412eae3e16b3c5f51782` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:13.081920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c43f5bdf9e412eae3e16b3c5f51782)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:13.083907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:13.144550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c43f5bdf9e412eae3e16b3c5f51782', 'request_id': 'req-cab944fd-1a54-4c5b-b92d-8a23bcc77705'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.154121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c43f5bdf9e412eae3e16b3c5f51782', 'request_id': 'req-cab944fd-1a54-4c5b-b92d-8a23bcc77705'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.162480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c43f5bdf9e412eae3e16b3c5f51782', 'request_id': 'req-cab944fd-1a54-4c5b-b92d-8a23bcc77705'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.174483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] The token cache is being invalidate because project c9c43f5bdf9e412eae3e16b3c5f51782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:13.174778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.175464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.177095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 971/1941] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:13 2022] DELETE /identity/v3/projects/c9c43f5bdf9e412eae3e16b3c5f51782 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:13.180949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866308c-f50c-45f6-ad5b-9680b79312cc service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:13.191216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866308c-f50c-45f6-ad5b-9680b79312cc service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:13.263777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:13.264943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:13.265248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:13.266155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] PATH_INFO: `/v3/users/eca4e990f0164d419744d4ae9ee84f33` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:13.267343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca4e990f0164d419744d4ae9ee84f33)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:13.270353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:13.323329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca4e990f0164d419744d4ae9ee84f33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.337053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca4e990f0164d419744d4ae9ee84f33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.337769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca4e990f0164d419744d4ae9ee84f33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.346198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca4e990f0164d419744d4ae9ee84f33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.346835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c504b14d-816b-404a-ab42-8437612e88f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca4e990f0164d419744d4ae9ee84f33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:13.360562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 971/1942] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:13 2022] DELETE /identity/v3/users/eca4e990f0164d419744d4ae9ee84f33 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:13.843684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:13.849753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:13.892117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab944fd-1a54-4c5b-b92d-8a23bcc77705 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:13.962247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:13.963323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:13.964200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:13.964486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] PATH_INFO: `/v3/projects/15368171f38647399b3be04008d3a0e4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:13.965713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15368171f38647399b3be04008d3a0e4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:13.967655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.019085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15368171f38647399b3be04008d3a0e4', 'request_id': 'req-06accc64-bd09-43bf-a12c-51d665b67391'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.037762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15368171f38647399b3be04008d3a0e4', 'request_id': 'req-06accc64-bd09-43bf-a12c-51d665b67391'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.045061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15368171f38647399b3be04008d3a0e4', 'request_id': 'req-06accc64-bd09-43bf-a12c-51d665b67391'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.055238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:14.058255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] The token cache is being invalidate because project 15368171f38647399b3be04008d3a0e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:14.058616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.059556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-06accc64-bd09-43bf-a12c-51d665b67391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.061638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 972/1943] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:13 2022] DELETE /identity/v3/projects/15368171f38647399b3be04008d3a0e4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:14.068121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a5fc-f35e-4da4-88fa-341b61e2928d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.068582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a5fc-f35e-4da4-88fa-341b61e2928d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.068944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a5fc-f35e-4da4-88fa-341b61e2928d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.092440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dd45a5fc-f35e-4da4-88fa-341b61e2928d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:14.094502 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dd45a5fc-f35e-4da4-88fa-341b61e2928d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:14.142059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:14.168404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 972/1944] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:14.183580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3d796-5b49-45b6-8aef-4128efdde9c1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.183878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3d796-5b49-45b6-8aef-4128efdde9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.184183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3d796-5b49-45b6-8aef-4128efdde9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.206290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cbb3d796-5b49-45b6-8aef-4128efdde9c1 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:14.207540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cbb3d796-5b49-45b6-8aef-4128efdde9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:14.220214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 973/1945] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:14.223549 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45a5fc-f35e-4da4-88fa-341b61e2928d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.233766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45a5fc-f35e-4da4-88fa-341b61e2928d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.238625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.239231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.239520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.239799 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.240581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:14.240890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.242663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.256114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 973/1946] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:14.260873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3d796-5b49-45b6-8aef-4128efdde9c1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.275191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3d796-5b49-45b6-8aef-4128efdde9c1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.281004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce518f-89dc-45c8-a2f2-c5a5db0a6dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.281759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce518f-89dc-45c8-a2f2-c5a5db0a6dae admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.282044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce518f-89dc-45c8-a2f2-c5a5db0a6dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.282318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce518f-89dc-45c8-a2f2-c5a5db0a6dae admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.283010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce518f-89dc-45c8-a2f2-c5a5db0a6dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.284964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce518f-89dc-45c8-a2f2-c5a5db0a6dae admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.313960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 974/1947] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:14 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:14.317404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.329518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28580f8-deab-4cb2-9815-1771e4cc70ff admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.396216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48434b6-8a44-4a3b-8fd5-905a243095cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.396821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48434b6-8a44-4a3b-8fd5-905a243095cb admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.397104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48434b6-8a44-4a3b-8fd5-905a243095cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.397380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48434b6-8a44-4a3b-8fd5-905a243095cb admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.398094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48434b6-8a44-4a3b-8fd5-905a243095cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.400037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48434b6-8a44-4a3b-8fd5-905a243095cb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.429857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 974/1948] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:14 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:14.431067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce518f-89dc-45c8-a2f2-c5a5db0a6dae admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.444503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce518f-89dc-45c8-a2f2-c5a5db0a6dae admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.449844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3d92e-266c-4001-bb4f-406c09368abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.450596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3d92e-266c-4001-bb4f-406c09368abd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.450976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3d92e-266c-4001-bb4f-406c09368abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.451329 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3d92e-266c-4001-bb4f-406c09368abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.452134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3d92e-266c-4001-bb4f-406c09368abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.454482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3d92e-266c-4001-bb4f-406c09368abd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.470748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 975/1949] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:14.476028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48434b6-8a44-4a3b-8fd5-905a243095cb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.487783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48434b6-8a44-4a3b-8fd5-905a243095cb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.493669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.494240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.494524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.494823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.495582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.497291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.512342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 975/1950] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:14.516374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3d92e-266c-4001-bb4f-406c09368abd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.521209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.523968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:14.527013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3d92e-266c-4001-bb4f-406c09368abd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.531868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15416db0-ee7e-4358-8e64-714a47eb24d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.532946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416db0-ee7e-4358-8e64-714a47eb24d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.533232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416db0-ee7e-4358-8e64-714a47eb24d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.533516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416db0-ee7e-4358-8e64-714a47eb24d5 admin admin] PATH_INFO: `/v3/projects/0682800620e64e488592b820129c69c5/users/0496d46f8a914504b210ed2d1b22ddd8/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.537651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f5e3a-66cd-4708-b76a-15bb173a4c5b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.546717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.548069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.548423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.548707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] PATH_INFO: `/v3/users/b8b45ca6d5cd4cc4b7ef359845e9f7de` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.550015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b45ca6d5cd4cc4b7ef359845e9f7de)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.552138 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.561544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15416db0-ee7e-4358-8e64-714a47eb24d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0682800620e64e488592b820129c69c5, user_id=0496d46f8a914504b210ed2d1b22ddd8, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.565149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15416db0-ee7e-4358-8e64-714a47eb24d5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.588613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 976/1951] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:14 2022] PUT /identity/v3/projects/0682800620e64e488592b820129c69c5/users/0496d46f8a914504b210ed2d1b22ddd8/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:14.604264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a81dd-e77a-454b-a19d-13d59f833fde None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.604709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a81dd-e77a-454b-a19d-13d59f833fde None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.605062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a81dd-e77a-454b-a19d-13d59f833fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.622387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b45ca6d5cd4cc4b7ef359845e9f7de'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.637167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b45ca6d5cd4cc4b7ef359845e9f7de'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.637449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-196a81dd-e77a-454b-a19d-13d59f833fde None None] MFA Rules not processed for user `0496d46f8a914504b210ed2d1b22ddd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:14.637896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b45ca6d5cd4cc4b7ef359845e9f7de'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.639160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-196a81dd-e77a-454b-a19d-13d59f833fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:14.646491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b45ca6d5cd4cc4b7ef359845e9f7de'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.647378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-944af27e-fa1c-4027-94a6-5ff9a272a29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b45ca6d5cd4cc4b7ef359845e9f7de'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:14.662263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 976/1952] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:14 2022] DELETE /identity/v3/users/b8b45ca6d5cd4cc4b7ef359845e9f7de => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:14.666595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adb65d-87d0-4e5b-aa99-843da8e6b1a8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.666893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adb65d-87d0-4e5b-aa99-843da8e6b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.667186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adb65d-87d0-4e5b-aa99-843da8e6b1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.685216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-66adb65d-87d0-4e5b-aa99-843da8e6b1a8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:14.686448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-66adb65d-87d0-4e5b-aa99-843da8e6b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:14.700837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:14.713054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 977/1953] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:14.725068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a81dd-e77a-454b-a19d-13d59f833fde None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.726515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-196a81dd-e77a-454b-a19d-13d59f833fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:14.756305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 977/1954] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:14.765892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adb65d-87d0-4e5b-aa99-843da8e6b1a8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:14.767265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-66adb65d-87d0-4e5b-aa99-843da8e6b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:14.789537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a81dd-e77a-454b-a19d-13d59f833fde None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.827194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adb65d-87d0-4e5b-aa99-843da8e6b1a8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:14.832219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.833102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.833515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.833848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.842089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:14.846433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.847922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.861225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 978/1955] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:05:14.869519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44ae18-d70b-4b48-858c-9d67e4aa66b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:14.870806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ae18-d70b-4b48-858c-9d67e4aa66b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:14.871257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ae18-d70b-4b48-858c-9d67e4aa66b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:14.871733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ae18-d70b-4b48-858c-9d67e4aa66b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:14.946215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44ae18-d70b-4b48-858c-9d67e4aa66b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:14.948210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44ae18-d70b-4b48-858c-9d67e4aa66b0 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:14.960043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 978/1956] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:05:15.304521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:15.306123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:15.322199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5b246-d3a5-40c0-a9f9-b36e3b94a2b6 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:15.327644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:15.329136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:15.329406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:15.329679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] PATH_INFO: `/v3/projects/454f733e042b460e9db049dff931b839` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:15.331199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=454f733e042b460e9db049dff931b839)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:15.333357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:15.394327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454f733e042b460e9db049dff931b839', 'request_id': 'req-87a501ba-8141-49d9-8f3a-d35014c58e94'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.401343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454f733e042b460e9db049dff931b839', 'request_id': 'req-87a501ba-8141-49d9-8f3a-d35014c58e94'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.407665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454f733e042b460e9db049dff931b839', 'request_id': 'req-87a501ba-8141-49d9-8f3a-d35014c58e94'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.419419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] The token cache is being invalidate because project 454f733e042b460e9db049dff931b839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:15.419716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.420354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.422526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 979/1957] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:15 2022] DELETE /identity/v3/projects/454f733e042b460e9db049dff931b839 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:15.426866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44ae18-d70b-4b48-858c-9d67e4aa66b0 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:15.438077 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44ae18-d70b-4b48-858c-9d67e4aa66b0 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:15.495016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:15.495890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:15.496199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:15.496511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] PATH_INFO: `/v3/users/54ac2fdc92214636a8f238e1fee2b9a0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:15.497707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ac2fdc92214636a8f238e1fee2b9a0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:15.499737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:15.547584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ac2fdc92214636a8f238e1fee2b9a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.559547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ac2fdc92214636a8f238e1fee2b9a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.560276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ac2fdc92214636a8f238e1fee2b9a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.561070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ac2fdc92214636a8f238e1fee2b9a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.568424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0832b874-1a58-45f2-9230-0dfa357e6e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ac2fdc92214636a8f238e1fee2b9a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:15.583859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 979/1958] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:15 2022] DELETE /identity/v3/users/54ac2fdc92214636a8f238e1fee2b9a0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:15.857865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:15.859407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:15.872899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a501ba-8141-49d9-8f3a-d35014c58e94 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:15.927824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:15.942892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:15.943770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:15.944058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:15.944374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] PATH_INFO: `/v3/projects/bf59d9654e9e4f5787da03c6a36a76c9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:15.945570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf59d9654e9e4f5787da03c6a36a76c9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:15.947514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:15.999160 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf59d9654e9e4f5787da03c6a36a76c9', 'request_id': 'req-3d24714a-5f21-437a-9eb2-1ed6cb68e731'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:16.016373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf59d9654e9e4f5787da03c6a36a76c9', 'request_id': 'req-3d24714a-5f21-437a-9eb2-1ed6cb68e731'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:16.022768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf59d9654e9e4f5787da03c6a36a76c9', 'request_id': 'req-3d24714a-5f21-437a-9eb2-1ed6cb68e731'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:16.037678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] The token cache is being invalidate because project bf59d9654e9e4f5787da03c6a36a76c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:16.038105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:16.038738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d24714a-5f21-437a-9eb2-1ed6cb68e731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:16.040324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 980/1959] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:15 2022] DELETE /identity/v3/projects/bf59d9654e9e4f5787da03c6a36a76c9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:16.045347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5f87a-78be-46db-aebd-d26d02f2836c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.045805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5f87a-78be-46db-aebd-d26d02f2836c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.045912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5f87a-78be-46db-aebd-d26d02f2836c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.067418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-27f5f87a-78be-46db-aebd-d26d02f2836c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:16.069064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-27f5f87a-78be-46db-aebd-d26d02f2836c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:16.080382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:16.144590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 980/1960] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:16.154305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46663c43-8af3-472f-a87d-29ab1916ec99 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.154841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46663c43-8af3-472f-a87d-29ab1916ec99 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.155328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46663c43-8af3-472f-a87d-29ab1916ec99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.181109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-46663c43-8af3-472f-a87d-29ab1916ec99 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:16.182459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-46663c43-8af3-472f-a87d-29ab1916ec99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:16.197836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 981/1961] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:16.198790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5f87a-78be-46db-aebd-d26d02f2836c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:16.210023 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5f87a-78be-46db-aebd-d26d02f2836c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:16.217038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:16.217727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.218011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.218291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.219395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:16.219711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:16.221432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:16.238566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 981/1962] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 12:05:16.242128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46663c43-8af3-472f-a87d-29ab1916ec99 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:16.255202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46663c43-8af3-472f-a87d-29ab1916ec99 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:16.261162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f9b1a-39b7-43fb-bf32-23e1bf5275cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:16.261843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f9b1a-39b7-43fb-bf32-23e1bf5275cd admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.262129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f9b1a-39b7-43fb-bf32-23e1bf5275cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.262411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f9b1a-39b7-43fb-bf32-23e1bf5275cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.263123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f9b1a-39b7-43fb-bf32-23e1bf5275cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:16.266468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f9b1a-39b7-43fb-bf32-23e1bf5275cd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:16.313598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 982/1963] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:16 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:16.314782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:16.325161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527d530-94f5-408a-bdc2-2e64ee3c6963 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:16.413997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60708a-dd86-471d-8d15-a1527b9e5c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:16.414785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60708a-dd86-471d-8d15-a1527b9e5c4f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.415140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60708a-dd86-471d-8d15-a1527b9e5c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.415487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60708a-dd86-471d-8d15-a1527b9e5c4f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.416294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60708a-dd86-471d-8d15-a1527b9e5c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:16.418287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60708a-dd86-471d-8d15-a1527b9e5c4f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:16.444131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 982/1964] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:16 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:16.445604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f9b1a-39b7-43fb-bf32-23e1bf5275cd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:16.457260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f9b1a-39b7-43fb-bf32-23e1bf5275cd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:16.462265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf02e9-e857-49bf-ac33-3fe4d3c552ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:16.462887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf02e9-e857-49bf-ac33-3fe4d3c552ef admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.463261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf02e9-e857-49bf-ac33-3fe4d3c552ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.463576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf02e9-e857-49bf-ac33-3fe4d3c552ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.464386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf02e9-e857-49bf-ac33-3fe4d3c552ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:16.466185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf02e9-e857-49bf-ac33-3fe4d3c552ef admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:16.480843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 983/1965] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:16.482139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60708a-dd86-471d-8d15-a1527b9e5c4f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:16.492694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60708a-dd86-471d-8d15-a1527b9e5c4f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:16.500658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7721c9-bf72-458e-b7ae-4e3965414f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:16.502269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7721c9-bf72-458e-b7ae-4e3965414f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.502622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7721c9-bf72-458e-b7ae-4e3965414f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.502907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7721c9-bf72-458e-b7ae-4e3965414f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.503660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7721c9-bf72-458e-b7ae-4e3965414f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:16.506788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7721c9-bf72-458e-b7ae-4e3965414f5f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:16.525112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 983/1966] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:16.530577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf02e9-e857-49bf-ac33-3fe4d3c552ef admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:16.545027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf02e9-e857-49bf-ac33-3fe4d3c552ef admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:16.550902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:16.551794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.552072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.552740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] PATH_INFO: `/v3/projects/07b8be51ccf34966a4361c48dc731c87/users/be3c35102e4d4a84b345354018456448/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.573052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b8be51ccf34966a4361c48dc731c87, user_id=be3c35102e4d4a84b345354018456448, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:16.575947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:16.598650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 984/1967] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:16 2022] PUT /identity/v3/projects/07b8be51ccf34966a4361c48dc731c87/users/be3c35102e4d4a84b345354018456448/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:16.609940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eb013-625a-4eb6-ae25-58d63d27ed45 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.610469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eb013-625a-4eb6-ae25-58d63d27ed45 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.610750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eb013-625a-4eb6-ae25-58d63d27ed45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.645168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-179eb013-625a-4eb6-ae25-58d63d27ed45 None None] MFA Rules not processed for user `be3c35102e4d4a84b345354018456448`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:16.646838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-179eb013-625a-4eb6-ae25-58d63d27ed45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:16.724479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 984/1968] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:16.736078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:16.741762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:16.808215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea647782-6725-4209-b0af-bb4ff3cb7e80 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:16.813477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f558e7-7b4d-41cf-b713-a1e08487508f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:16.814341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f558e7-7b4d-41cf-b713-a1e08487508f service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.814626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f558e7-7b4d-41cf-b713-a1e08487508f service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.814904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f558e7-7b4d-41cf-b713-a1e08487508f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.825098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f558e7-7b4d-41cf-b713-a1e08487508f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:16.826600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f558e7-7b4d-41cf-b713-a1e08487508f service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:16.838433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 985/1969] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:16.858271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eb013-625a-4eb6-ae25-58d63d27ed45 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:16.872472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:16.944401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eb013-625a-4eb6-ae25-58d63d27ed45 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:16.950189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:16.951175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:16.951526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:16.951867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] PATH_INFO: `/v3/users/be3c35102e4d4a84b345354018456448` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:16.953256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3c35102e4d4a84b345354018456448)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:16.955760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:17.018203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3c35102e4d4a84b345354018456448'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.031082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3c35102e4d4a84b345354018456448'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.032273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3c35102e4d4a84b345354018456448'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.033025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3c35102e4d4a84b345354018456448'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.040964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3c35102e4d4a84b345354018456448'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.058804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 985/1970] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:16 2022] DELETE /identity/v3/users/be3c35102e4d4a84b345354018456448 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:17.065132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292b1c-b586-44ee-8a4d-e477c95674be None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:17.065429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292b1c-b586-44ee-8a4d-e477c95674be None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:17.065763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292b1c-b586-44ee-8a4d-e477c95674be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:17.090285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-48292b1c-b586-44ee-8a4d-e477c95674be None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:17.091963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-48292b1c-b586-44ee-8a4d-e477c95674be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:17.105782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:17.174918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 986/1971] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:17.189645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:17.191117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:17.257194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a72c32-a4b9-4143-8467-a993ba7fadc7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:17.261194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe59454-a726-4746-88be-e7789a6334a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:17.261946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe59454-a726-4746-88be-e7789a6334a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:17.262230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe59454-a726-4746-88be-e7789a6334a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:17.262524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe59454-a726-4746-88be-e7789a6334a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:17.273305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe59454-a726-4746-88be-e7789a6334a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:17.274937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe59454-a726-4746-88be-e7789a6334a3 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:17.286322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 986/1972] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:17.596261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48292b1c-b586-44ee-8a4d-e477c95674be None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:17.598489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-48292b1c-b586-44ee-8a4d-e477c95674be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:17.614994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48292b1c-b586-44ee-8a4d-e477c95674be None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:17.620061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:17.620993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:17.621288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:17.621570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] PATH_INFO: `/v3/projects/07b8be51ccf34966a4361c48dc731c87` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:17.622794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b8be51ccf34966a4361c48dc731c87)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:17.624999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:17.683426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b8be51ccf34966a4361c48dc731c87', 'request_id': 'req-d57c4bcf-e211-4b76-b3ec-500c27188d33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.691415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b8be51ccf34966a4361c48dc731c87', 'request_id': 'req-d57c4bcf-e211-4b76-b3ec-500c27188d33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.697427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b8be51ccf34966a4361c48dc731c87', 'request_id': 'req-d57c4bcf-e211-4b76-b3ec-500c27188d33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.711975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] The token cache is being invalidate because project 07b8be51ccf34966a4361c48dc731c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:17.712311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.712968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d57c4bcf-e211-4b76-b3ec-500c27188d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:17.717703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 987/1973] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:17 2022] DELETE /identity/v3/projects/07b8be51ccf34966a4361c48dc731c87 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:17.719748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a111ee-8539-4de7-bece-8878c5ae9c2d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:17.720220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a111ee-8539-4de7-bece-8878c5ae9c2d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:17.720581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a111ee-8539-4de7-bece-8878c5ae9c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:17.740922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e9a111ee-8539-4de7-bece-8878c5ae9c2d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:17.742224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e9a111ee-8539-4de7-bece-8878c5ae9c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:17.775683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:17.813959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 987/1974] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:17.822138 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490a7b5-7312-4f3b-bd35-37a28fad1c2e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:17.822433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490a7b5-7312-4f3b-bd35-37a28fad1c2e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:17.822718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490a7b5-7312-4f3b-bd35-37a28fad1c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:17.841578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c490a7b5-7312-4f3b-bd35-37a28fad1c2e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:17.842801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c490a7b5-7312-4f3b-bd35-37a28fad1c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:17.851299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 988/1975] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:17.855481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a111ee-8539-4de7-bece-8878c5ae9c2d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:17.866055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a111ee-8539-4de7-bece-8878c5ae9c2d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:17.871071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:17.871753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:17.872045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:17.872356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:17.873095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:17.873434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:17.875786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:17.888107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 988/1976] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:17.892576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490a7b5-7312-4f3b-bd35-37a28fad1c2e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:17.902338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490a7b5-7312-4f3b-bd35-37a28fad1c2e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:17.908282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90e70b-b011-4e15-a445-e343accbe8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:17.908964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90e70b-b011-4e15-a445-e343accbe8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:17.909288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90e70b-b011-4e15-a445-e343accbe8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:17.909527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90e70b-b011-4e15-a445-e343accbe8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:17.910228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90e70b-b011-4e15-a445-e343accbe8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:17.912184 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90e70b-b011-4e15-a445-e343accbe8a8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:17.949782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 989/1977] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:17 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:17.950896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:17.961032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435b0cb-62cb-41b3-854c-eb4a76b3336c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.019688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab3abd-6e33-4f9f-8fee-d4d5a8c5424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.020680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3abd-6e33-4f9f-8fee-d4d5a8c5424f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.020968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3abd-6e33-4f9f-8fee-d4d5a8c5424f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.021254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab3abd-6e33-4f9f-8fee-d4d5a8c5424f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.021946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab3abd-6e33-4f9f-8fee-d4d5a8c5424f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.023890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab3abd-6e33-4f9f-8fee-d4d5a8c5424f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.049866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 989/1978] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:17 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:18.051374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90e70b-b011-4e15-a445-e343accbe8a8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.062658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90e70b-b011-4e15-a445-e343accbe8a8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.067989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e61e0-88fd-43d7-a692-7db335ad3141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.068938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e61e0-88fd-43d7-a692-7db335ad3141 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.069226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e61e0-88fd-43d7-a692-7db335ad3141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.069498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e61e0-88fd-43d7-a692-7db335ad3141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.070243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e61e0-88fd-43d7-a692-7db335ad3141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.071862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e61e0-88fd-43d7-a692-7db335ad3141 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.083887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 990/1979] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:18.087880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab3abd-6e33-4f9f-8fee-d4d5a8c5424f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.099910 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab3abd-6e33-4f9f-8fee-d4d5a8c5424f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.101898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ab4ba-8c8c-419a-901f-bdfc5f34a4f8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.102140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ab4ba-8c8c-419a-901f-bdfc5f34a4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.102483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ab4ba-8c8c-419a-901f-bdfc5f34a4f8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.104660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 991/1980] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 12:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:05:18.112128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd17f3b-fff8-4b98-bcbd-c6038ecbc784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.115090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7d7df-0f56-429c-8456-9f32848f1b56 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.115388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7d7df-0f56-429c-8456-9f32848f1b56 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.115675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7d7df-0f56-429c-8456-9f32848f1b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.117492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd17f3b-fff8-4b98-bcbd-c6038ecbc784 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.117833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd17f3b-fff8-4b98-bcbd-c6038ecbc784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.118122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd17f3b-fff8-4b98-bcbd-c6038ecbc784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.119424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd17f3b-fff8-4b98-bcbd-c6038ecbc784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.121123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd17f3b-fff8-4b98-bcbd-c6038ecbc784 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.143362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-62b7d7df-0f56-429c-8456-9f32848f1b56 None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:18.144616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-62b7d7df-0f56-429c-8456-9f32848f1b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.148574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 990/1981] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:18.152199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd17f3b-fff8-4b98-bcbd-c6038ecbc784 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.170425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd17f3b-fff8-4b98-bcbd-c6038ecbc784 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.177234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.177695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.177847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.178188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] PATH_INFO: `/v3/projects/c91b2d3653664c0096993e03b170dce9/users/0dd74dfe3e304922a34dc0f032c16221/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.203239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91b2d3653664c0096993e03b170dce9, user_id=0dd74dfe3e304922a34dc0f032c16221, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.208102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.219845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:18.224607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 991/1982] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:18 2022] PUT /identity/v3/projects/c91b2d3653664c0096993e03b170dce9/users/0dd74dfe3e304922a34dc0f032c16221/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:18.225703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.227498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.244089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 992/1983] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:18.245051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7d7df-0f56-429c-8456-9f32848f1b56 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.246372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-62b7d7df-0f56-429c-8456-9f32848f1b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.261027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7d7df-0f56-429c-8456-9f32848f1b56 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.301581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90425bfb-8a9f-4aa5-8f93-73ea50c10387 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.306940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.307850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.308134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.308443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] PATH_INFO: `/v3/users/e591526a8c3645de9c5965a40f8ef90a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.309753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e591526a8c3645de9c5965a40f8ef90a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.311588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.321128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.321997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.322291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.322570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] PATH_INFO: `/v3/users/0496d46f8a914504b210ed2d1b22ddd8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.323778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0496d46f8a914504b210ed2d1b22ddd8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.326043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.328420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:18.365376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e591526a8c3645de9c5965a40f8ef90a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.376793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0496d46f8a914504b210ed2d1b22ddd8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.377205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e591526a8c3645de9c5965a40f8ef90a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.377902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e591526a8c3645de9c5965a40f8ef90a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.378609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e591526a8c3645de9c5965a40f8ef90a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.385909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df77ea72-fb64-46d4-95ce-12c0fbd290d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e591526a8c3645de9c5965a40f8ef90a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.389170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0496d46f8a914504b210ed2d1b22ddd8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.390073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0496d46f8a914504b210ed2d1b22ddd8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.399174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0496d46f8a914504b210ed2d1b22ddd8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.399865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0496d46f8a914504b210ed2d1b22ddd8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.403822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 992/1984] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:18 2022] DELETE /identity/v3/users/e591526a8c3645de9c5965a40f8ef90a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:18.403822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33be7-dccb-4790-aa86-d5f8a57b6a3b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.403822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33be7-dccb-4790-aa86-d5f8a57b6a3b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.403822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33be7-dccb-4790-aa86-d5f8a57b6a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.428609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 993/1985] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:18 2022] DELETE /identity/v3/users/0496d46f8a914504b210ed2d1b22ddd8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:18.429706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.431030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.434860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e5c33be7-dccb-4790-aa86-d5f8a57b6a3b None None] MFA Rules not processed for user `0dd74dfe3e304922a34dc0f032c16221`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:18.435643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e5c33be7-dccb-4790-aa86-d5f8a57b6a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.517970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea877f0-e473-4538-bca1-52d3d3a81e25 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.524671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.525469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.525758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.526087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.526655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 993/1986] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:18.529055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328926f8-839c-41d0-81be-679e7f8a9486 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.529549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328926f8-839c-41d0-81be-679e7f8a9486 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.529837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328926f8-839c-41d0-81be-679e7f8a9486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.534383 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.557811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.559690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.566417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-328926f8-839c-41d0-81be-679e7f8a9486 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:18.567622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-328926f8-839c-41d0-81be-679e7f8a9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.575531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 994/1987] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:18.575531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.575531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.582509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:18.647106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 994/1988] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:18.657596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328926f8-839c-41d0-81be-679e7f8a9486 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.659497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-328926f8-839c-41d0-81be-679e7f8a9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.673123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49082e95-8dbb-4ce2-86ac-078611db74f5 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.677495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.678524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.678850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.679151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.691612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.692083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:18.693243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.709792 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 995/1989] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:18.720075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.721577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.736942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa0668-b4ab-49a9-95c6-99d8453f43d6 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.737353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328926f8-839c-41d0-81be-679e7f8a9486 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.741258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.742000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.742100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.742406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.742742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.742875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.743235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.743521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] PATH_INFO: `/v3/projects/a60b241805384ef49352b3791cfd3d8c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.744825 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60b241805384ef49352b3791cfd3d8c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.746746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.754104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.755638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.765958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 995/1990] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:05:18.767120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:18.768826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.786519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962c928-7af3-4c25-b86f-0fcadb64508d service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:18.791679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:18.793018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.793761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.793761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] PATH_INFO: `/v3/users/0dd74dfe3e304922a34dc0f032c16221` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.795044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dd74dfe3e304922a34dc0f032c16221)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:18.798224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:18.806109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60b241805384ef49352b3791cfd3d8c', 'request_id': 'req-ab5c386e-81fb-47b1-b16d-b64eca81d164'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.813960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60b241805384ef49352b3791cfd3d8c', 'request_id': 'req-ab5c386e-81fb-47b1-b16d-b64eca81d164'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.820701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60b241805384ef49352b3791cfd3d8c', 'request_id': 'req-ab5c386e-81fb-47b1-b16d-b64eca81d164'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.839243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] The token cache is being invalidate because project a60b241805384ef49352b3791cfd3d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:18.839541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.840573 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ab5c386e-81fb-47b1-b16d-b64eca81d164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.842872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 996/1991] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:18 2022] DELETE /identity/v3/projects/a60b241805384ef49352b3791cfd3d8c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:18.849254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cce88-f4cd-40c1-84df-f31b2ad4ce8b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.849775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cce88-f4cd-40c1-84df-f31b2ad4ce8b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.850292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cce88-f4cd-40c1-84df-f31b2ad4ce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.864599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dd74dfe3e304922a34dc0f032c16221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.872979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-563cce88-f4cd-40c1-84df-f31b2ad4ce8b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:18.875918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-563cce88-f4cd-40c1-84df-f31b2ad4ce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.880672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dd74dfe3e304922a34dc0f032c16221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.881483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dd74dfe3e304922a34dc0f032c16221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.882277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dd74dfe3e304922a34dc0f032c16221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.895289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5baf614b-3e2b-4202-9c83-74ddc4beb45e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dd74dfe3e304922a34dc0f032c16221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:18.911487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 996/1992] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:18 2022] DELETE /identity/v3/users/0dd74dfe3e304922a34dc0f032c16221 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:18.916715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3954aa9-8c83-4357-b28b-7b00b5d17d5b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.917079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3954aa9-8c83-4357-b28b-7b00b5d17d5b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.917560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3954aa9-8c83-4357-b28b-7b00b5d17d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.948866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f3954aa9-8c83-4357-b28b-7b00b5d17d5b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:18.950709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f3954aa9-8c83-4357-b28b-7b00b5d17d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:18.954297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 997/1993] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:18.962910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa56f8-7dd5-46b0-affb-b1016516b7fc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:18.963557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa56f8-7dd5-46b0-affb-b1016516b7fc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:18.964039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa56f8-7dd5-46b0-affb-b1016516b7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:18.992129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c7fa56f8-7dd5-46b0-affb-b1016516b7fc None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:18.993407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c7fa56f8-7dd5-46b0-affb-b1016516b7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.018572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 997/1994] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:19.028998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:19.029561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3954aa9-8c83-4357-b28b-7b00b5d17d5b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.031056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f3954aa9-8c83-4357-b28b-7b00b5d17d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.073771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 998/1995] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:19.075029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa56f8-7dd5-46b0-affb-b1016516b7fc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.077063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c7fa56f8-7dd5-46b0-affb-b1016516b7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.092593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa56f8-7dd5-46b0-affb-b1016516b7fc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.097396 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:19.097938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.098976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.099277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.099687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] PATH_INFO: `/v3/projects/0682800620e64e488592b820129c69c5` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.101189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0682800620e64e488592b820129c69c5)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.103186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.114945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3954aa9-8c83-4357-b28b-7b00b5d17d5b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.119105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38a67-c7c8-4127-9224-ab6ebf64034e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.119838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38a67-c7c8-4127-9224-ab6ebf64034e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.120256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38a67-c7c8-4127-9224-ab6ebf64034e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.120550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38a67-c7c8-4127-9224-ab6ebf64034e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.130840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38a67-c7c8-4127-9224-ab6ebf64034e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.132259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38a67-c7c8-4127-9224-ab6ebf64034e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.142871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 998/1996] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:05:19.143500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38a67-c7c8-4127-9224-ab6ebf64034e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.153860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38a67-c7c8-4127-9224-ab6ebf64034e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.155363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0682800620e64e488592b820129c69c5', 'request_id': 'req-8e07fcf5-3f4e-4a89-9f05-749c896c9267'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.162982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0682800620e64e488592b820129c69c5', 'request_id': 'req-8e07fcf5-3f4e-4a89-9f05-749c896c9267'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.173025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0682800620e64e488592b820129c69c5', 'request_id': 'req-8e07fcf5-3f4e-4a89-9f05-749c896c9267'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.192405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] The token cache is being invalidate because project 0682800620e64e488592b820129c69c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:19.192753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.193375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8e07fcf5-3f4e-4a89-9f05-749c896c9267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.196276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 999/1997] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:19 2022] DELETE /identity/v3/projects/0682800620e64e488592b820129c69c5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:19.201829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ce22-ab92-4bb4-a53b-355691bf8591 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.204424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ce22-ab92-4bb4-a53b-355691bf8591 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.206062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ce22-ab92-4bb4-a53b-355691bf8591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.235779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.236536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.236823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.237096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.237834 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:19.238459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.239984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0be9ce22-ab92-4bb4-a53b-355691bf8591 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:19.240104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.241061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0be9ce22-ab92-4bb4-a53b-355691bf8591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.250129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1000/1998] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:19.255171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 999/1999] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:19.257277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.257750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c5be-d875-4b38-b536-4ae7df97fe0e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.258124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c5be-d875-4b38-b536-4ae7df97fe0e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.258373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c5be-d875-4b38-b536-4ae7df97fe0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.259237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.273331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0503b-9c93-4b75-9b28-782090606fa4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.278400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c23739-ecb9-4d44-b760-3f29b3fbdf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.279378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23739-ecb9-4d44-b760-3f29b3fbdf82 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.279495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d662c5be-d875-4b38-b536-4ae7df97fe0e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:19.279603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23739-ecb9-4d44-b760-3f29b3fbdf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.279603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c23739-ecb9-4d44-b760-3f29b3fbdf82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.280350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c23739-ecb9-4d44-b760-3f29b3fbdf82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.280506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d662c5be-d875-4b38-b536-4ae7df97fe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.282857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c23739-ecb9-4d44-b760-3f29b3fbdf82 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.292949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1001/2000] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:19.293850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662c5be-d875-4b38-b536-4ae7df97fe0e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.304574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662c5be-d875-4b38-b536-4ae7df97fe0e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.309695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.310405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.310694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.311063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.311704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:19.312021 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.314421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.319360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1000/2001] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:19.320535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c23739-ecb9-4d44-b760-3f29b3fbdf82 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.337648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1002/2002] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:19.341697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.353102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3314e-ab70-458a-81e7-43b494fd698a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.400913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c23739-ecb9-4d44-b760-3f29b3fbdf82 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.406532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315f779-00f1-4d8a-a783-02faeadbe89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.407298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315f779-00f1-4d8a-a783-02faeadbe89f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.407667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315f779-00f1-4d8a-a783-02faeadbe89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.408016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315f779-00f1-4d8a-a783-02faeadbe89f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.408814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315f779-00f1-4d8a-a783-02faeadbe89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.411229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315f779-00f1-4d8a-a783-02faeadbe89f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.426300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.426984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.427271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.427557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.428321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.430482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.436192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1001/2003] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:19.439125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315f779-00f1-4d8a-a783-02faeadbe89f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.451314 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315f779-00f1-4d8a-a783-02faeadbe89f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.457589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69593-ff36-476e-97cb-34895f6f363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.458370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69593-ff36-476e-97cb-34895f6f363f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.458694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69593-ff36-476e-97cb-34895f6f363f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.459051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69593-ff36-476e-97cb-34895f6f363f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.459844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e69593-ff36-476e-97cb-34895f6f363f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.461652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e69593-ff36-476e-97cb-34895f6f363f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.471358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1003/2004] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:19.472476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.476597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1002/2005] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:19.477344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69593-ff36-476e-97cb-34895f6f363f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.478048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.488186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69593-ff36-476e-97cb-34895f6f363f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.495627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:19.521987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:19.551587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7aefb-a7c6-4395-927c-e964a4eb6d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.552301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7aefb-a7c6-4395-927c-e964a4eb6d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.552590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7aefb-a7c6-4395-927c-e964a4eb6d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.552736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63bc7-a293-4e47-a053-131f05ec51c7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.552855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7aefb-a7c6-4395-927c-e964a4eb6d9d admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.553557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7aefb-a7c6-4395-927c-e964a4eb6d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.555495 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7aefb-a7c6-4395-927c-e964a4eb6d9d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.557370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.558248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.558537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.558818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] PATH_INFO: `/v3/projects/c91b2d3653664c0096993e03b170dce9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.560013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c91b2d3653664c0096993e03b170dce9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.561963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.576865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1003/2006] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:19.577717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7aefb-a7c6-4395-927c-e964a4eb6d9d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.587740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7aefb-a7c6-4395-927c-e964a4eb6d9d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.592876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f32bb1-eaae-4294-b308-c102a78b9f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.594186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f32bb1-eaae-4294-b308-c102a78b9f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.594468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f32bb1-eaae-4294-b308-c102a78b9f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.594754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f32bb1-eaae-4294-b308-c102a78b9f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.595492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f32bb1-eaae-4294-b308-c102a78b9f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.597142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f32bb1-eaae-4294-b308-c102a78b9f21 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.609681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1004/2007] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:19.610694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f32bb1-eaae-4294-b308-c102a78b9f21 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.622418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f32bb1-eaae-4294-b308-c102a78b9f21 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.624407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c91b2d3653664c0096993e03b170dce9', 'request_id': 'req-b3d88a5a-412f-4d72-9edc-17877b430d88'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.631644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c91b2d3653664c0096993e03b170dce9', 'request_id': 'req-b3d88a5a-412f-4d72-9edc-17877b430d88'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.637457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c91b2d3653664c0096993e03b170dce9', 'request_id': 'req-b3d88a5a-412f-4d72-9edc-17877b430d88'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.652040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] The token cache is being invalidate because project c91b2d3653664c0096993e03b170dce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:19.652364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.653027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:19.654657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1004/2008] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:19 2022] DELETE /identity/v3/projects/c91b2d3653664c0096993e03b170dce9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:19.659439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.660850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.685583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d77e6-c555-4a31-ad6f-1b3fa3b30be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.686268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d77e6-c555-4a31-ad6f-1b3fa3b30be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.686561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d77e6-c555-4a31-ad6f-1b3fa3b30be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.686841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d77e6-c555-4a31-ad6f-1b3fa3b30be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.688130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d77e6-c555-4a31-ad6f-1b3fa3b30be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.689779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d77e6-c555-4a31-ad6f-1b3fa3b30be5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.699684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1005/2009] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:19.700987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91771a14-4eee-4b72-b57c-6c97b025aaea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.701285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91771a14-4eee-4b72-b57c-6c97b025aaea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.701573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91771a14-4eee-4b72-b57c-6c97b025aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.720241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-91771a14-4eee-4b72-b57c-6c97b025aaea None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:19.721447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-91771a14-4eee-4b72-b57c-6c97b025aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.728400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d88a5a-412f-4d72-9edc-17877b430d88 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.730518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1006/2010] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:19.737021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91771a14-4eee-4b72-b57c-6c97b025aaea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.737021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-91771a14-4eee-4b72-b57c-6c97b025aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.738700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df358-cd5c-4faf-8cc9-edc676b06143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.739617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df358-cd5c-4faf-8cc9-edc676b06143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.739924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df358-cd5c-4faf-8cc9-edc676b06143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.740243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0df358-cd5c-4faf-8cc9-edc676b06143 admin admin] PATH_INFO: `/v3/projects/e7b01b4dbe234da4abb82077d84bb238/users/8cba74531efe4959ad40e0390f1609aa/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.744564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91771a14-4eee-4b72-b57c-6c97b025aaea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.749236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf0741-75fe-4055-b130-ba1bdf680d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.750434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf0741-75fe-4055-b130-ba1bdf680d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.750722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf0741-75fe-4055-b130-ba1bdf680d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.750997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf0741-75fe-4055-b130-ba1bdf680d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.751737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf0741-75fe-4055-b130-ba1bdf680d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.754740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf0741-75fe-4055-b130-ba1bdf680d03 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.757837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0df358-cd5c-4faf-8cc9-edc676b06143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b01b4dbe234da4abb82077d84bb238, user_id=8cba74531efe4959ad40e0390f1609aa, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.760619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0df358-cd5c-4faf-8cc9-edc676b06143 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.767619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1007/2011] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:19.769104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccaff-b66a-4afa-84f4-700981ccf4be None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.769399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccaff-b66a-4afa-84f4-700981ccf4be None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.769927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccaff-b66a-4afa-84f4-700981ccf4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.780361 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1005/2012] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:19 2022] PUT /identity/v3/projects/e7b01b4dbe234da4abb82077d84bb238/users/8cba74531efe4959ad40e0390f1609aa/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:19.781469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df358-cd5c-4faf-8cc9-edc676b06143 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.800267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-023ccaff-b66a-4afa-84f4-700981ccf4be None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:19.801293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-023ccaff-b66a-4afa-84f4-700981ccf4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.815753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:19.858017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0df358-cd5c-4faf-8cc9-edc676b06143 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.865482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99b225-d931-429f-9aeb-d7185bd4af4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.866382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b225-d931-429f-9aeb-d7185bd4af4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.866669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b225-d931-429f-9aeb-d7185bd4af4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.866949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b225-d931-429f-9aeb-d7185bd4af4f admin admin] PATH_INFO: `/v3/projects/b7843d3fb7cb402b8a85179da7bfffaa/users/a440b40bc3124bafa502f66a004d0de0/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.873244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1008/2013] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:19.874635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda701b3-9006-4d07-9293-6c88650853b1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.875683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda701b3-9006-4d07-9293-6c88650853b1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.876137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda701b3-9006-4d07-9293-6c88650853b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.889337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b225-d931-429f-9aeb-d7185bd4af4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7843d3fb7cb402b8a85179da7bfffaa, user_id=a440b40bc3124bafa502f66a004d0de0, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.892534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b225-d931-429f-9aeb-d7185bd4af4f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.909826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1006/2014] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:19 2022] PUT /identity/v3/projects/b7843d3fb7cb402b8a85179da7bfffaa/users/a440b40bc3124bafa502f66a004d0de0/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:19.910523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99b225-d931-429f-9aeb-d7185bd4af4f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:19.912133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cda701b3-9006-4d07-9293-6c88650853b1 None None] MFA Rules not processed for user `8cba74531efe4959ad40e0390f1609aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:19.913515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cda701b3-9006-4d07-9293-6c88650853b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:19.936041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:19.979795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99b225-d931-429f-9aeb-d7185bd4af4f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:19.985219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:19.985909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.986193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.986472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:19.987209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:19.987519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:19.989147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:19.994839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1009/2015] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:19.995968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd20f8-495e-40e1-8fda-0a8e232e36a5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:19.996283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd20f8-495e-40e1-8fda-0a8e232e36a5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:19.996571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd20f8-495e-40e1-8fda-0a8e232e36a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.002504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1007/2016] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:20.006812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.016913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb68917-f906-4fe5-9196-d9158cac86e7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.022003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.022174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f5fd20f8-495e-40e1-8fda-0a8e232e36a5 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:20.022664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.022954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.023238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.023374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f5fd20f8-495e-40e1-8fda-0a8e232e36a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:20.023961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.026035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.031881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1010/2017] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:20.032884 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd20f8-495e-40e1-8fda-0a8e232e36a5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.034178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f5fd20f8-495e-40e1-8fda-0a8e232e36a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:20.059935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1008/2018] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:20 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:20.059935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.059935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:20.116005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd20f8-495e-40e1-8fda-0a8e232e36a5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.117092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36277268-3b0b-4a78-9f87-2af95c6abe01 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.121345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae666804-9d02-43dc-9a65-9886c3cd3947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.122007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666804-9d02-43dc-9a65-9886c3cd3947 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.122291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666804-9d02-43dc-9a65-9886c3cd3947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.122572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae666804-9d02-43dc-9a65-9886c3cd3947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.162169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:20.191034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca632563-8b9f-4f0a-a645-7886cffae592 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.191717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca632563-8b9f-4f0a-a645-7886cffae592 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.192180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca632563-8b9f-4f0a-a645-7886cffae592 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.192555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca632563-8b9f-4f0a-a645-7886cffae592 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.192849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae666804-9d02-43dc-9a65-9886c3cd3947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.194333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae666804-9d02-43dc-9a65-9886c3cd3947 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.204775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1009/2019] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:20.205647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae666804-9d02-43dc-9a65-9886c3cd3947 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.214839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae666804-9d02-43dc-9a65-9886c3cd3947 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.220301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa20af-bbd4-4cda-9861-231e666bae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.220988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa20af-bbd4-4cda-9861-231e666bae0c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.221273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa20af-bbd4-4cda-9861-231e666bae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.221556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa20af-bbd4-4cda-9861-231e666bae0c admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.222282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa20af-bbd4-4cda-9861-231e666bae0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.224376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa20af-bbd4-4cda-9861-231e666bae0c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.255959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:20.256741 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1010/2020] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:20 2022] POST /identity/v3/users => generated 292 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:20.257732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa20af-bbd4-4cda-9861-231e666bae0c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.267773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca632563-8b9f-4f0a-a645-7886cffae592 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.269009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa20af-bbd4-4cda-9861-231e666bae0c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.270479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca632563-8b9f-4f0a-a645-7886cffae592 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.274401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4a50f-9a36-4787-b586-baa134ec663a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.275011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a50f-9a36-4787-b586-baa134ec663a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.275424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a50f-9a36-4787-b586-baa134ec663a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.275749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a50f-9a36-4787-b586-baa134ec663a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.277070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4a50f-9a36-4787-b586-baa134ec663a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.278774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4a50f-9a36-4787-b586-baa134ec663a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.287471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1011/2021] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:20.291605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1011/2022] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:20.295391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca632563-8b9f-4f0a-a645-7886cffae592 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.308989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca632563-8b9f-4f0a-a645-7886cffae592 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.315711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883608ed-bdf7-4265-a0fa-eff7fc51127e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.316580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883608ed-bdf7-4265-a0fa-eff7fc51127e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.316937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883608ed-bdf7-4265-a0fa-eff7fc51127e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.317290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883608ed-bdf7-4265-a0fa-eff7fc51127e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.318090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883608ed-bdf7-4265-a0fa-eff7fc51127e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.320373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883608ed-bdf7-4265-a0fa-eff7fc51127e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.336631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1012/2023] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:20.341839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4a50f-9a36-4787-b586-baa134ec663a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.360851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4a50f-9a36-4787-b586-baa134ec663a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.365894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be2d30-d550-415d-926e-f30dd8fd116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.366807 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be2d30-d550-415d-926e-f30dd8fd116e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.367090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be2d30-d550-415d-926e-f30dd8fd116e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.367396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be2d30-d550-415d-926e-f30dd8fd116e admin admin] PATH_INFO: `/v3/projects/dac5c0d03225472cbfbc93b02b55ca8c/users/f01e1d0fb7f34a509d5dcc1473a07d20/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.385737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be2d30-d550-415d-926e-f30dd8fd116e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac5c0d03225472cbfbc93b02b55ca8c, user_id=f01e1d0fb7f34a509d5dcc1473a07d20, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.389115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be2d30-d550-415d-926e-f30dd8fd116e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.409092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1012/2024] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:20 2022] PUT /identity/v3/projects/dac5c0d03225472cbfbc93b02b55ca8c/users/f01e1d0fb7f34a509d5dcc1473a07d20/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:20.419494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42671a7-edd6-4a17-9a60-c3d31bb19f1b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.419702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42671a7-edd6-4a17-9a60-c3d31bb19f1b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.419981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42671a7-edd6-4a17-9a60-c3d31bb19f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.452175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b42671a7-edd6-4a17-9a60-c3d31bb19f1b None None] MFA Rules not processed for user `f01e1d0fb7f34a509d5dcc1473a07d20`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:20.453962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b42671a7-edd6-4a17-9a60-c3d31bb19f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:20.540852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1013/2025] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:20.545817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be2d30-d550-415d-926e-f30dd8fd116e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.640047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be2d30-d550-415d-926e-f30dd8fd116e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.647074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf260c0c-88db-4d92-a802-fdedfd28081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.648297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf260c0c-88db-4d92-a802-fdedfd28081f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.648665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf260c0c-88db-4d92-a802-fdedfd28081f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.649057 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf260c0c-88db-4d92-a802-fdedfd28081f admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.651222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf260c0c-88db-4d92-a802-fdedfd28081f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.652140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf260c0c-88db-4d92-a802-fdedfd28081f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.702568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1013/2026] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:20 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:20.708298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42671a7-edd6-4a17-9a60-c3d31bb19f1b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.721583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42671a7-edd6-4a17-9a60-c3d31bb19f1b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.808478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabeab8-01d2-494e-9038-16e1dedaf943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.809257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabeab8-01d2-494e-9038-16e1dedaf943 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.809643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabeab8-01d2-494e-9038-16e1dedaf943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.809993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabeab8-01d2-494e-9038-16e1dedaf943 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.810775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabeab8-01d2-494e-9038-16e1dedaf943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.813875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabeab8-01d2-494e-9038-16e1dedaf943 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.836789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1014/2027] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:20 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:20.840776 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf260c0c-88db-4d92-a802-fdedfd28081f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.851356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf260c0c-88db-4d92-a802-fdedfd28081f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.856096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e03dd-b024-44c4-9d3d-d78b88d3f5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.856801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e03dd-b024-44c4-9d3d-d78b88d3f5cc admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.857087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e03dd-b024-44c4-9d3d-d78b88d3f5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.857373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e03dd-b024-44c4-9d3d-d78b88d3f5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.858236 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e03dd-b024-44c4-9d3d-d78b88d3f5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.860415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e03dd-b024-44c4-9d3d-d78b88d3f5cc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.872828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1014/2028] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:20.877738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabeab8-01d2-494e-9038-16e1dedaf943 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.891416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabeab8-01d2-494e-9038-16e1dedaf943 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.896818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aafca-7bf7-4f1f-b435-56557b525784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.897498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aafca-7bf7-4f1f-b435-56557b525784 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.897812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aafca-7bf7-4f1f-b435-56557b525784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.898071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aafca-7bf7-4f1f-b435-56557b525784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.898813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aafca-7bf7-4f1f-b435-56557b525784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.900569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aafca-7bf7-4f1f-b435-56557b525784 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.911906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1015/2029] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:20.915992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e03dd-b024-44c4-9d3d-d78b88d3f5cc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:20.927212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e03dd-b024-44c4-9d3d-d78b88d3f5cc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:20.932024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a2465-628f-4912-b783-a231d13f802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:20.932986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a2465-628f-4912-b783-a231d13f802c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.933364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a2465-628f-4912-b783-a231d13f802c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.933558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a2465-628f-4912-b783-a231d13f802c admin admin] PATH_INFO: `/v3/projects/41a6ff3d1bb74eceadb43668704d8609/users/442c00f7bef049bc94e6fa2c7ee9d16a/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:20.949683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a2465-628f-4912-b783-a231d13f802c admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a6ff3d1bb74eceadb43668704d8609, user_id=442c00f7bef049bc94e6fa2c7ee9d16a, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:20.952423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a2465-628f-4912-b783-a231d13f802c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:20.972284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1015/2030] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:20 2022] PUT /identity/v3/projects/41a6ff3d1bb74eceadb43668704d8609/users/442c00f7bef049bc94e6fa2c7ee9d16a/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:20.983354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4dda4-1ec8-4352-911f-95a9fd0217c8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:20.983632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4dda4-1ec8-4352-911f-95a9fd0217c8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:20.983925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4dda4-1ec8-4352-911f-95a9fd0217c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.019087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609989d-1a1e-4a8d-8ef8-191e413e5957 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.020092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609989d-1a1e-4a8d-8ef8-191e413e5957 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.020474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609989d-1a1e-4a8d-8ef8-191e413e5957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.022017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-85c4dda4-1ec8-4352-911f-95a9fd0217c8 None None] MFA Rules not processed for user `442c00f7bef049bc94e6fa2c7ee9d16a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:21.023292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-85c4dda4-1ec8-4352-911f-95a9fd0217c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:21.058705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6609989d-1a1e-4a8d-8ef8-191e413e5957 None None] MFA Rules not processed for user `a440b40bc3124bafa502f66a004d0de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:21.060214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6609989d-1a1e-4a8d-8ef8-191e413e5957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:21.070926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:21.086645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:21.129615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1016/2031] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:21.134337 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4dda4-1ec8-4352-911f-95a9fd0217c8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:21.166231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1016/2032] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:21.167509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609989d-1a1e-4a8d-8ef8-191e413e5957 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:21.169778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6609989d-1a1e-4a8d-8ef8-191e413e5957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:21.254104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4dda4-1ec8-4352-911f-95a9fd0217c8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:21.257263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609989d-1a1e-4a8d-8ef8-191e413e5957 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:21.259878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bba56f-04e4-4eb4-b9e8-e0ebf6626376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:21.260978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bba56f-04e4-4eb4-b9e8-e0ebf6626376 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.261203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bba56f-04e4-4eb4-b9e8-e0ebf6626376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.261594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bba56f-04e4-4eb4-b9e8-e0ebf6626376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.261894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086df71-c015-43ea-9173-95759f521759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:21.262377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bba56f-04e4-4eb4-b9e8-e0ebf6626376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:21.262674 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086df71-c015-43ea-9173-95759f521759 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.263041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086df71-c015-43ea-9173-95759f521759 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.263389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086df71-c015-43ea-9173-95759f521759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.264593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bba56f-04e4-4eb4-b9e8-e0ebf6626376 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:21.275319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086df71-c015-43ea-9173-95759f521759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:21.276521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086df71-c015-43ea-9173-95759f521759 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:21.287882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1017/2033] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:21.295814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1017/2034] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:21 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:21.300211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086df71-c015-43ea-9173-95759f521759 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:21.311459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086df71-c015-43ea-9173-95759f521759 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:21.393885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806303f-5ed6-4291-b840-59fa5e77054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:21.394578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806303f-5ed6-4291-b840-59fa5e77054f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.395136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806303f-5ed6-4291-b840-59fa5e77054f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.395418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806303f-5ed6-4291-b840-59fa5e77054f admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.396187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806303f-5ed6-4291-b840-59fa5e77054f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:21.398086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806303f-5ed6-4291-b840-59fa5e77054f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:21.425775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1018/2035] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:21 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:21.426976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bba56f-04e4-4eb4-b9e8-e0ebf6626376 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:21.438948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bba56f-04e4-4eb4-b9e8-e0ebf6626376 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:21.444573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ecd06-4d50-40e0-9f1e-37ebfdaea162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:21.445468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ecd06-4d50-40e0-9f1e-37ebfdaea162 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.445824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ecd06-4d50-40e0-9f1e-37ebfdaea162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.446214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ecd06-4d50-40e0-9f1e-37ebfdaea162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.447027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ecd06-4d50-40e0-9f1e-37ebfdaea162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:21.448755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ecd06-4d50-40e0-9f1e-37ebfdaea162 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:21.464197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1018/2036] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:21.467965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806303f-5ed6-4291-b840-59fa5e77054f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:21.483124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806303f-5ed6-4291-b840-59fa5e77054f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:21.488916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf04f9-6b95-40e1-84b7-8e905c60e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:21.490125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf04f9-6b95-40e1-84b7-8e905c60e654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.490491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf04f9-6b95-40e1-84b7-8e905c60e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.490881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf04f9-6b95-40e1-84b7-8e905c60e654 admin admin] PATH_INFO: `/v3/projects/a047765440d14856a5887a392ab8f991/users/c26ed397260a4b1d9089f102034d8435/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.512116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf04f9-6b95-40e1-84b7-8e905c60e654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a047765440d14856a5887a392ab8f991, user_id=c26ed397260a4b1d9089f102034d8435, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:21.517257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf04f9-6b95-40e1-84b7-8e905c60e654 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:21.533599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1019/2037] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:21 2022] PUT /identity/v3/projects/a047765440d14856a5887a392ab8f991/users/c26ed397260a4b1d9089f102034d8435/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:21.537312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ecd06-4d50-40e0-9f1e-37ebfdaea162 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:21.551523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ecd06-4d50-40e0-9f1e-37ebfdaea162 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:21.678548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556715a8-8b8f-435f-bdf1-8dc354006fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:21.679326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556715a8-8b8f-435f-bdf1-8dc354006fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.679681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556715a8-8b8f-435f-bdf1-8dc354006fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.680030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556715a8-8b8f-435f-bdf1-8dc354006fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.680870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556715a8-8b8f-435f-bdf1-8dc354006fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:21.682777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556715a8-8b8f-435f-bdf1-8dc354006fa7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:21.696218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1019/2038] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:21.700136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf04f9-6b95-40e1-84b7-8e905c60e654 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:21.711717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf04f9-6b95-40e1-84b7-8e905c60e654 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:21.720969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:21.721950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.722306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.722712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] PATH_INFO: `/v3/projects/a047765440d14856a5887a392ab8f991/users/c26ed397260a4b1d9089f102034d8435/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.724938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a047765440d14856a5887a392ab8f991, user_id=c26ed397260a4b1d9089f102034d8435, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:21.729670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:21.743318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1020/2039] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:21 2022] PUT /identity/v3/projects/a047765440d14856a5887a392ab8f991/users/c26ed397260a4b1d9089f102034d8435/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:21.753944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299a64c-58fe-43ae-8e8f-ec237f086182 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.754244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299a64c-58fe-43ae-8e8f-ec237f086182 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.754557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299a64c-58fe-43ae-8e8f-ec237f086182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:21.789219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3299a64c-58fe-43ae-8e8f-ec237f086182 None None] MFA Rules not processed for user `c26ed397260a4b1d9089f102034d8435`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:21.790408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3299a64c-58fe-43ae-8e8f-ec237f086182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:21.827402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:21.866954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1020/2040] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:21.879178 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:21.881189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:21.896311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:21.944637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f240a61-0aff-4a20-b109-e39cf3d3d636 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:21.948852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:21.949461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:21.949737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:21.950068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:22.015203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:22.017279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:22.029864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1021/2041] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:22.759678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a876e70-4717-4e5f-b95d-cbaacd7a38b2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:22.760267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a876e70-4717-4e5f-b95d-cbaacd7a38b2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:22.760267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a876e70-4717-4e5f-b95d-cbaacd7a38b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:22.780327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8a876e70-4717-4e5f-b95d-cbaacd7a38b2 None None] MFA Rules not processed for user `a440b40bc3124bafa502f66a004d0de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:22.782239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8a876e70-4717-4e5f-b95d-cbaacd7a38b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:22.798545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:22.799940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:22.838759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1021/2042] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:22.868127 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59565ee1-976c-4f34-b943-aea75151321e service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:22.873943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:22.875429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:22.875429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:22.875429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] PATH_INFO: `/v3/users/6d8b74cde0bd4e5d9dca41253f26af48` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:22.876694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d8b74cde0bd4e5d9dca41253f26af48)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:22.878687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:22.933115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d8b74cde0bd4e5d9dca41253f26af48'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:22.945358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d8b74cde0bd4e5d9dca41253f26af48'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:22.946082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d8b74cde0bd4e5d9dca41253f26af48'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:22.951349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:22.956152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d8b74cde0bd4e5d9dca41253f26af48'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:22.957088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d8b74cde0bd4e5d9dca41253f26af48'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:22.977017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1022/2043] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:22 2022] DELETE /identity/v3/users/6d8b74cde0bd4e5d9dca41253f26af48 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:22.982152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f4b7-b59a-47c5-a183-8607767e34b9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:22.983179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f4b7-b59a-47c5-a183-8607767e34b9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:22.983179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f4b7-b59a-47c5-a183-8607767e34b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:23.003253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-80e7f4b7-b59a-47c5-a183-8607767e34b9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:23.004548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-80e7f4b7-b59a-47c5-a183-8607767e34b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:23.075152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1022/2044] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:23.085818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:23.087241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:23.146991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820300e-7645-418f-b8a4-2334948907d5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:23.150785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177b0d5-148d-4441-8a81-78682bb2bf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:23.151774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177b0d5-148d-4441-8a81-78682bb2bf09 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:23.152086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177b0d5-148d-4441-8a81-78682bb2bf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:23.152415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177b0d5-148d-4441-8a81-78682bb2bf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:23.163184 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177b0d5-148d-4441-8a81-78682bb2bf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:23.164678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177b0d5-148d-4441-8a81-78682bb2bf09 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:23.175959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1023/2045] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:23.536353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7f4b7-b59a-47c5-a183-8607767e34b9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:23.538497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-80e7f4b7-b59a-47c5-a183-8607767e34b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:23.565141 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7f4b7-b59a-47c5-a183-8607767e34b9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:23.570656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:23.576498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:23.576498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:23.576498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] PATH_INFO: `/v3/projects/529bcc9cae8841cf9d9948fd713defda` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:23.576498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=529bcc9cae8841cf9d9948fd713defda)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:23.576498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:23.638673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '529bcc9cae8841cf9d9948fd713defda', 'request_id': 'req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:23.645369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '529bcc9cae8841cf9d9948fd713defda', 'request_id': 'req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:23.652481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '529bcc9cae8841cf9d9948fd713defda', 'request_id': 'req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:23.664711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:23.668896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] The token cache is being invalidate because project 529bcc9cae8841cf9d9948fd713defda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:23.669490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:23.670611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6fcc04d3-e9e3-4b47-b83f-ae8aa1794f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:23.672836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1023/2046] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:23 2022] DELETE /identity/v3/projects/529bcc9cae8841cf9d9948fd713defda => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:23.677788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ad87c-695c-43ac-b61c-6a0a6c322798 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:23.678272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ad87c-695c-43ac-b61c-6a0a6c322798 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:23.678749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ad87c-695c-43ac-b61c-6a0a6c322798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:23.698312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-657ad87c-695c-43ac-b61c-6a0a6c322798 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:23.699464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-657ad87c-695c-43ac-b61c-6a0a6c322798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:23.766372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1024/2047] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:23.774589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218443f9-fb73-4077-8637-3f13c52f03c9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:23.775428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218443f9-fb73-4077-8637-3f13c52f03c9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:23.775738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218443f9-fb73-4077-8637-3f13c52f03c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:23.795147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-218443f9-fb73-4077-8637-3f13c52f03c9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:23.796936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-218443f9-fb73-4077-8637-3f13c52f03c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:23.805710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1024/2048] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:23.809828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ad87c-695c-43ac-b61c-6a0a6c322798 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:23.820886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ad87c-695c-43ac-b61c-6a0a6c322798 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:23.825661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:23.826330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:23.826615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:23.826894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:23.827646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:23.827960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:23.829636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:23.841506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1025/2049] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:23.846117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218443f9-fb73-4077-8637-3f13c52f03c9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:23.855479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218443f9-fb73-4077-8637-3f13c52f03c9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:23.861162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02393e-1d5e-49a0-956f-5b27caf5c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:23.861823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02393e-1d5e-49a0-956f-5b27caf5c5eb admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:23.862114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02393e-1d5e-49a0-956f-5b27caf5c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:23.862433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02393e-1d5e-49a0-956f-5b27caf5c5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:23.863130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02393e-1d5e-49a0-956f-5b27caf5c5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:23.865119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02393e-1d5e-49a0-956f-5b27caf5c5eb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:23.894189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1025/2050] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:23 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:23.898616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:23.909283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d813832-153a-481e-91a9-905f2bf435bf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:23.965416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67326361-727c-45f0-a3cb-b4228106a562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:23.966089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67326361-727c-45f0-a3cb-b4228106a562 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:23.966885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67326361-727c-45f0-a3cb-b4228106a562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:23.967177 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67326361-727c-45f0-a3cb-b4228106a562 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:23.967895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67326361-727c-45f0-a3cb-b4228106a562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:23.970694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67326361-727c-45f0-a3cb-b4228106a562 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:23.991754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1026/2051] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:23 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:23.995881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02393e-1d5e-49a0-956f-5b27caf5c5eb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.006507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02393e-1d5e-49a0-956f-5b27caf5c5eb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.011176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77eca0-0dff-46fe-b80e-2332d2d0ff9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.011840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77eca0-0dff-46fe-b80e-2332d2d0ff9f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.012130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77eca0-0dff-46fe-b80e-2332d2d0ff9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.012453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77eca0-0dff-46fe-b80e-2332d2d0ff9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.013185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77eca0-0dff-46fe-b80e-2332d2d0ff9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.014797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77eca0-0dff-46fe-b80e-2332d2d0ff9f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.028264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1026/2052] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:24.033112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67326361-727c-45f0-a3cb-b4228106a562 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.043109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67326361-727c-45f0-a3cb-b4228106a562 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.048671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d5f22-352a-4465-899e-b8d6c0081d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.049329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d5f22-352a-4465-899e-b8d6c0081d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.049612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d5f22-352a-4465-899e-b8d6c0081d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.049890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d5f22-352a-4465-899e-b8d6c0081d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.050610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d5f22-352a-4465-899e-b8d6c0081d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.052278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d5f22-352a-4465-899e-b8d6c0081d42 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.053940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:24.066365 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1027/2053] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:24.070463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77eca0-0dff-46fe-b80e-2332d2d0ff9f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.080568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77eca0-0dff-46fe-b80e-2332d2d0ff9f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.085194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc4806-a70d-44ef-8fc5-04c36da09471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.086095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4806-a70d-44ef-8fc5-04c36da09471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.086380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4806-a70d-44ef-8fc5-04c36da09471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.086657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4806-a70d-44ef-8fc5-04c36da09471 admin admin] PATH_INFO: `/v3/projects/91215efee3c34596bc62c56dc45c6e40/users/88ecb71e569b4f0ea393251984e1c1c1/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.103392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc4806-a70d-44ef-8fc5-04c36da09471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91215efee3c34596bc62c56dc45c6e40, user_id=88ecb71e569b4f0ea393251984e1c1c1, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.106110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc4806-a70d-44ef-8fc5-04c36da09471 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.119735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1027/2054] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:24 2022] PUT /identity/v3/projects/91215efee3c34596bc62c56dc45c6e40/users/88ecb71e569b4f0ea393251984e1c1c1/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:24.131040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902e9e-9d09-46e3-92a3-59e580b99c85 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.131334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902e9e-9d09-46e3-92a3-59e580b99c85 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.131617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a902e9e-9d09-46e3-92a3-59e580b99c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.159470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2a902e9e-9d09-46e3-92a3-59e580b99c85 None None] MFA Rules not processed for user `88ecb71e569b4f0ea393251984e1c1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:24.160877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2a902e9e-9d09-46e3-92a3-59e580b99c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:24.228766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1028/2055] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:24.233229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc4806-a70d-44ef-8fc5-04c36da09471 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.298443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc4806-a70d-44ef-8fc5-04c36da09471 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.303195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e3ced-c9f8-4361-9948-9449217e51a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.303860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3ced-c9f8-4361-9948-9449217e51a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.304148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3ced-c9f8-4361-9948-9449217e51a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.305046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e3ced-c9f8-4361-9948-9449217e51a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.305751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e3ced-c9f8-4361-9948-9449217e51a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.307682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e3ced-c9f8-4361-9948-9449217e51a5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.335718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1028/2056] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:24 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:24.339711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a902e9e-9d09-46e3-92a3-59e580b99c85 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.351346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a902e9e-9d09-46e3-92a3-59e580b99c85 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.409097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59020193-5900-4a68-b050-2cda8f475f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.409910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020193-5900-4a68-b050-2cda8f475f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.410540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020193-5900-4a68-b050-2cda8f475f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.410540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020193-5900-4a68-b050-2cda8f475f06 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.411142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020193-5900-4a68-b050-2cda8f475f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.413125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020193-5900-4a68-b050-2cda8f475f06 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.438701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1029/2057] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:24 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:24.439730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e3ced-c9f8-4361-9948-9449217e51a5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.450854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e3ced-c9f8-4361-9948-9449217e51a5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.456582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87da053-65a7-4c02-a484-913ac2903de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.457343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da053-65a7-4c02-a484-913ac2903de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.457732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da053-65a7-4c02-a484-913ac2903de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.458087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87da053-65a7-4c02-a484-913ac2903de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.458901 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87da053-65a7-4c02-a484-913ac2903de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.460611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87da053-65a7-4c02-a484-913ac2903de7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.472767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1029/2058] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:24.477418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59020193-5900-4a68-b050-2cda8f475f06 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.487525 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59020193-5900-4a68-b050-2cda8f475f06 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.492189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bfa8e-61ea-4d41-88c2-107b453e16dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.493070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bfa8e-61ea-4d41-88c2-107b453e16dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.493352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bfa8e-61ea-4d41-88c2-107b453e16dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.493630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bfa8e-61ea-4d41-88c2-107b453e16dc admin admin] PATH_INFO: `/v3/projects/4bd349f428c147bf88e84657c81ca0f9/users/8ec92edaebea412fb25e32edf4579b8c/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.508970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bfa8e-61ea-4d41-88c2-107b453e16dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd349f428c147bf88e84657c81ca0f9, user_id=8ec92edaebea412fb25e32edf4579b8c, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.512022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5bfa8e-61ea-4d41-88c2-107b453e16dc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.527133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1030/2059] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:24 2022] PUT /identity/v3/projects/4bd349f428c147bf88e84657c81ca0f9/users/8ec92edaebea412fb25e32edf4579b8c/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:24.532029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87da053-65a7-4c02-a484-913ac2903de7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.542643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87da053-65a7-4c02-a484-913ac2903de7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.620629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563944f-be74-4662-a6bb-f03ca733c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.621307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563944f-be74-4662-a6bb-f03ca733c9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.621606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563944f-be74-4662-a6bb-f03ca733c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.622259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563944f-be74-4662-a6bb-f03ca733c9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.622602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563944f-be74-4662-a6bb-f03ca733c9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.624243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563944f-be74-4662-a6bb-f03ca733c9fa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.634407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1030/2060] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:24.638656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bfa8e-61ea-4d41-88c2-107b453e16dc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.650560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bfa8e-61ea-4d41-88c2-107b453e16dc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.655100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.656585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.657009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.657285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] PATH_INFO: `/v3/projects/4bd349f428c147bf88e84657c81ca0f9/users/8ec92edaebea412fb25e32edf4579b8c/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.659381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd349f428c147bf88e84657c81ca0f9, user_id=8ec92edaebea412fb25e32edf4579b8c, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.662150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.673062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1031/2061] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:24 2022] PUT /identity/v3/projects/4bd349f428c147bf88e84657c81ca0f9/users/8ec92edaebea412fb25e32edf4579b8c/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:24.683747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a82a3-9b6e-49af-8c41-185e554641c0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.684044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a82a3-9b6e-49af-8c41-185e554641c0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.684371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a82a3-9b6e-49af-8c41-185e554641c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.700790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:24.716681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fa7a82a3-9b6e-49af-8c41-185e554641c0 None None] MFA Rules not processed for user `8ec92edaebea412fb25e32edf4579b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:24.717969 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fa7a82a3-9b6e-49af-8c41-185e554641c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:24.791225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1031/2062] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:24.802537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:24.804387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:24.850338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:24.869110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb68af-bcf2-4531-bd5f-36578d7ef7f6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:24.872848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a891c-d346-4d08-99b0-2f54c6801e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:24.873530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a891c-d346-4d08-99b0-2f54c6801e59 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:24.873891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a891c-d346-4d08-99b0-2f54c6801e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:24.874171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a891c-d346-4d08-99b0-2f54c6801e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:24.933373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a891c-d346-4d08-99b0-2f54c6801e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:24.934771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a891c-d346-4d08-99b0-2f54c6801e59 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:24.944932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1032/2063] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:26.782274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a82a3-9b6e-49af-8c41-185e554641c0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:26.783800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fa7a82a3-9b6e-49af-8c41-185e554641c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:26.796613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a82a3-9b6e-49af-8c41-185e554641c0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:26.800841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:26.801702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:26.801980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:26.802299 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:26.804145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:26.872656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:26.874508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:26.886067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1032/2064] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:27.511240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a891c-d346-4d08-99b0-2f54c6801e59 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:27.532897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a891c-d346-4d08-99b0-2f54c6801e59 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:27.536592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee68ab7-afee-428f-931f-b33aacbad0e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:27.537246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee68ab7-afee-428f-931f-b33aacbad0e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:27.537529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee68ab7-afee-428f-931f-b33aacbad0e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:27.538230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee68ab7-afee-428f-931f-b33aacbad0e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:27.539638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8ee68ab7-afee-428f-931f-b33aacbad0e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:27.620521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee68ab7-afee-428f-931f-b33aacbad0e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:27.622162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee68ab7-afee-428f-931f-b33aacbad0e4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:27.635735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1033/2065] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:27.710935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:27.779525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8ee75-6bf3-41bd-adb0-8139998898f2 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:27.784937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:27.785848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:27.786138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:27.786418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] PATH_INFO: `/v3/users/88ecb71e569b4f0ea393251984e1c1c1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:27.787653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88ecb71e569b4f0ea393251984e1c1c1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:27.789637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:27.824191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:27.837392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88ecb71e569b4f0ea393251984e1c1c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:27.850345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88ecb71e569b4f0ea393251984e1c1c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:27.850957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88ecb71e569b4f0ea393251984e1c1c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:27.851568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88ecb71e569b4f0ea393251984e1c1c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:27.859035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88ecb71e569b4f0ea393251984e1c1c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:27.871527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1033/2066] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:27 2022] DELETE /identity/v3/users/88ecb71e569b4f0ea393251984e1c1c1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:27.877558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a424ac-39e1-4586-a758-77159753054b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:27.877891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a424ac-39e1-4586-a758-77159753054b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:27.878243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a424ac-39e1-4586-a758-77159753054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:27.898219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-19a424ac-39e1-4586-a758-77159753054b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:27.899739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-19a424ac-39e1-4586-a758-77159753054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:27.977646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1034/2067] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:27.988130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:27.989663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:28.053414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2f6d8-55d6-4e1d-89c6-131e240e306b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:28.057298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78879cdb-e6fb-4a31-af3b-9bfdebdfa5e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:28.057983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78879cdb-e6fb-4a31-af3b-9bfdebdfa5e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:28.058302 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78879cdb-e6fb-4a31-af3b-9bfdebdfa5e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:28.058586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78879cdb-e6fb-4a31-af3b-9bfdebdfa5e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:28.069234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78879cdb-e6fb-4a31-af3b-9bfdebdfa5e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:28.070692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78879cdb-e6fb-4a31-af3b-9bfdebdfa5e7 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:28.080954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1034/2068] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:28.363661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a424ac-39e1-4586-a758-77159753054b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:28.365116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-19a424ac-39e1-4586-a758-77159753054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:28.378748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a424ac-39e1-4586-a758-77159753054b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:28.384608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:28.385854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:28.386291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:28.386614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] PATH_INFO: `/v3/projects/91215efee3c34596bc62c56dc45c6e40` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:28.387828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91215efee3c34596bc62c56dc45c6e40)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:28.389810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:28.414006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:28.438010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91215efee3c34596bc62c56dc45c6e40', 'request_id': 'req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.444051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91215efee3c34596bc62c56dc45c6e40', 'request_id': 'req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.449545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91215efee3c34596bc62c56dc45c6e40', 'request_id': 'req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.462300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] The token cache is being invalidate because project 91215efee3c34596bc62c56dc45c6e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:28.462596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.463225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.464815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1035/2069] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:28 2022] DELETE /identity/v3/projects/91215efee3c34596bc62c56dc45c6e40 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:28.469245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78879cdb-e6fb-4a31-af3b-9bfdebdfa5e7 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:28.479037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78879cdb-e6fb-4a31-af3b-9bfdebdfa5e7 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:28.535590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:28.536548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:28.536853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:28.537140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] PATH_INFO: `/v3/users/8ec92edaebea412fb25e32edf4579b8c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:28.538478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec92edaebea412fb25e32edf4579b8c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:28.540431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:28.588001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec92edaebea412fb25e32edf4579b8c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.595371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:28.604410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec92edaebea412fb25e32edf4579b8c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.605332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec92edaebea412fb25e32edf4579b8c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.605977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec92edaebea412fb25e32edf4579b8c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.613613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec92edaebea412fb25e32edf4579b8c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:28.626848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1035/2070] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:28 2022] DELETE /identity/v3/users/8ec92edaebea412fb25e32edf4579b8c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:28.862350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:28.865216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:28.883428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2622a5-3f53-429c-ac93-efe1bdd2ce10 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:28.958756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:28.961836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:28.966704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:28.968142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:28.969089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:28.969448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] PATH_INFO: `/v3/users/8cba74531efe4959ad40e0390f1609aa` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:28.971165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cba74531efe4959ad40e0390f1609aa)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:28.973795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:28.990692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e47af-1819-4c33-9034-c9696c51e74a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.003232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.004341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.005119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.005466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] PATH_INFO: `/v3/projects/4bd349f428c147bf88e84657c81ca0f9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.007163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd349f428c147bf88e84657c81ca0f9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.009948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.037355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cba74531efe4959ad40e0390f1609aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.054734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cba74531efe4959ad40e0390f1609aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.055691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cba74531efe4959ad40e0390f1609aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.068325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cba74531efe4959ad40e0390f1609aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.069031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1c06bbee-8546-461b-b0ff-d98a298de85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cba74531efe4959ad40e0390f1609aa'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.094200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd349f428c147bf88e84657c81ca0f9', 'request_id': 'req-a78cc897-29f6-4e02-ad3b-4f47987e4f51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.108390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd349f428c147bf88e84657c81ca0f9', 'request_id': 'req-a78cc897-29f6-4e02-ad3b-4f47987e4f51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.110576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1036/2071] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:28 2022] DELETE /identity/v3/users/8cba74531efe4959ad40e0390f1609aa => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:29.112966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4ab7a-148d-4be5-abf6-48bfb29c9c28 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.115530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4ab7a-148d-4be5-abf6-48bfb29c9c28 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.117145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd349f428c147bf88e84657c81ca0f9', 'request_id': 'req-a78cc897-29f6-4e02-ad3b-4f47987e4f51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.117685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4ab7a-148d-4be5-abf6-48bfb29c9c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.131048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] The token cache is being invalidate because project 4bd349f428c147bf88e84657c81ca0f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:29.131343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.132109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a78cc897-29f6-4e02-ad3b-4f47987e4f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.136870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1036/2072] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:28 2022] DELETE /identity/v3/projects/4bd349f428c147bf88e84657c81ca0f9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:29.144909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1ca58-2ba4-46f3-acc3-3c0fa1c44339 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.145409 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1ca58-2ba4-46f3-acc3-3c0fa1c44339 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.145799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1ca58-2ba4-46f3-acc3-3c0fa1c44339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.163538 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-31f4ab7a-148d-4be5-abf6-48bfb29c9c28 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:29.165319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-31f4ab7a-148d-4be5-abf6-48bfb29c9c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:29.172357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-59f1ca58-2ba4-46f3-acc3-3c0fa1c44339 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:29.173856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-59f1ca58-2ba4-46f3-acc3-3c0fa1c44339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:29.234636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:29.281200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1037/2073] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:29.288829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f4381-9bc3-4f40-a64a-7a9f5d88cad0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.289423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f4381-9bc3-4f40-a64a-7a9f5d88cad0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.289784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f4381-9bc3-4f40-a64a-7a9f5d88cad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.316205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1037/2074] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:29.316395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a86f4381-9bc3-4f40-a64a-7a9f5d88cad0 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:29.317019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a86f4381-9bc3-4f40-a64a-7a9f5d88cad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:29.330346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1038/2075] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:29.337716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4ab7a-148d-4be5-abf6-48bfb29c9c28 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.338022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f4381-9bc3-4f40-a64a-7a9f5d88cad0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.339905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-31f4ab7a-148d-4be5-abf6-48bfb29c9c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:29.352952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f4381-9bc3-4f40-a64a-7a9f5d88cad0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.367632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.367632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.367957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.368413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.368929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:29.369266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.370942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.393637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1039/2076] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 12:05:29.395079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.409336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6180f-b4cb-4215-bb5e-252251ee0407 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.416092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a21129-264c-4bbc-81b8-0a7f5b6c702f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.416959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21129-264c-4bbc-81b8-0a7f5b6c702f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.417458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21129-264c-4bbc-81b8-0a7f5b6c702f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.417808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21129-264c-4bbc-81b8-0a7f5b6c702f admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.418581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a21129-264c-4bbc-81b8-0a7f5b6c702f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.420660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a21129-264c-4bbc-81b8-0a7f5b6c702f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.421890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4ab7a-148d-4be5-abf6-48bfb29c9c28 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.426419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ecfd7-c2ec-41c1-b4df-b3d0126ceb65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.427762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ecfd7-c2ec-41c1-b4df-b3d0126ceb65 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.428123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ecfd7-c2ec-41c1-b4df-b3d0126ceb65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.428499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ecfd7-c2ec-41c1-b4df-b3d0126ceb65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.440530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ecfd7-c2ec-41c1-b4df-b3d0126ceb65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.442189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ecfd7-c2ec-41c1-b4df-b3d0126ceb65 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.454126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1038/2077] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:29.459935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1040/2078] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:29 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:29.463922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ecfd7-c2ec-41c1-b4df-b3d0126ceb65 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.476478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ecfd7-c2ec-41c1-b4df-b3d0126ceb65 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.545597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fe4c1-9448-4a27-8fc4-f2bc467a7d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.547098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fe4c1-9448-4a27-8fc4-f2bc467a7d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.547705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fe4c1-9448-4a27-8fc4-f2bc467a7d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.548104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fe4c1-9448-4a27-8fc4-f2bc467a7d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.548920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fe4c1-9448-4a27-8fc4-f2bc467a7d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.552359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fe4c1-9448-4a27-8fc4-f2bc467a7d3a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.573296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1039/2079] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:29 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:29.577342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a21129-264c-4bbc-81b8-0a7f5b6c702f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.589405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a21129-264c-4bbc-81b8-0a7f5b6c702f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.597297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ce6e3-b11b-4447-a47e-68bd9dc32365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.597983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ce6e3-b11b-4447-a47e-68bd9dc32365 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.598274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ce6e3-b11b-4447-a47e-68bd9dc32365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.598564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ce6e3-b11b-4447-a47e-68bd9dc32365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.599367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ce6e3-b11b-4447-a47e-68bd9dc32365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.601003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ce6e3-b11b-4447-a47e-68bd9dc32365 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.612999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1041/2080] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:29.616883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fe4c1-9448-4a27-8fc4-f2bc467a7d3a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.629223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fe4c1-9448-4a27-8fc4-f2bc467a7d3a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.634027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1561eae-1dfd-4c4e-b8e6-16da75b1477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.634707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561eae-1dfd-4c4e-b8e6-16da75b1477a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.634997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561eae-1dfd-4c4e-b8e6-16da75b1477a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.635283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561eae-1dfd-4c4e-b8e6-16da75b1477a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.636020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1561eae-1dfd-4c4e-b8e6-16da75b1477a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.637699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1561eae-1dfd-4c4e-b8e6-16da75b1477a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.649517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1040/2081] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:29.654043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ce6e3-b11b-4447-a47e-68bd9dc32365 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.659773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:29.666373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ce6e3-b11b-4447-a47e-68bd9dc32365 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.673699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.674594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.674878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.680184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] PATH_INFO: `/v3/projects/eec44ec24e8f4194b4812dacf57bbfe9/users/2cf75e4a8c934c59aeac173052a84653/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.701341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec44ec24e8f4194b4812dacf57bbfe9, user_id=2cf75e4a8c934c59aeac173052a84653, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.705816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.723230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1042/2082] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:29 2022] PUT /identity/v3/projects/eec44ec24e8f4194b4812dacf57bbfe9/users/2cf75e4a8c934c59aeac173052a84653/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:29.735772 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494406-73a0-44fc-834a-ad75c692907e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.736062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494406-73a0-44fc-834a-ad75c692907e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.736374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494406-73a0-44fc-834a-ad75c692907e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.767929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c1494406-73a0-44fc-834a-ad75c692907e None None] MFA Rules not processed for user `2cf75e4a8c934c59aeac173052a84653`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:29.769225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c1494406-73a0-44fc-834a-ad75c692907e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:29.777116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.779574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:29.815836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:29.842786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1041/2083] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:29.854913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494406-73a0-44fc-834a-ad75c692907e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:29.856368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c1494406-73a0-44fc-834a-ad75c692907e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:29.860681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f095d34-fc69-44fa-b358-660cae4bdc54 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.865580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.866445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.866730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.867228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] PATH_INFO: `/v3/projects/e7b01b4dbe234da4abb82077d84bb238` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.868233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b01b4dbe234da4abb82077d84bb238)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:29.870157 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:29.917527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494406-73a0-44fc-834a-ad75c692907e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:29.927888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b01b4dbe234da4abb82077d84bb238', 'request_id': 'req-68450461-44d8-49fa-9bd0-f7621ca543dc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.938324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b01b4dbe234da4abb82077d84bb238', 'request_id': 'req-68450461-44d8-49fa-9bd0-f7621ca543dc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.945001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b01b4dbe234da4abb82077d84bb238', 'request_id': 'req-68450461-44d8-49fa-9bd0-f7621ca543dc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.959314 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] The token cache is being invalidate because project e7b01b4dbe234da4abb82077d84bb238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:29.959918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.960459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68450461-44d8-49fa-9bd0-f7621ca543dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:29.962255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1043/2084] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:29 2022] DELETE /identity/v3/projects/e7b01b4dbe234da4abb82077d84bb238 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:29.966981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c618b8-5c4f-4b59-86d7-21ed8114801f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.967213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c618b8-5c4f-4b59-86d7-21ed8114801f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.967500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c618b8-5c4f-4b59-86d7-21ed8114801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.976638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad37bdce39f64836b0cd14613cabe91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:29.977294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:29.977584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:29.977875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:29.979728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:29.985966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-90c618b8-5c4f-4b59-86d7-21ed8114801f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:29.987160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90c618b8-5c4f-4b59-86d7-21ed8114801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:30.043490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:30.044971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:30.053681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1044/2085] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:30.058468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1042/2086] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:05:30.064379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12dac-9eb3-4363-a85e-19dca75d406f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.064729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12dac-9eb3-4363-a85e-19dca75d406f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.064942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12dac-9eb3-4363-a85e-19dca75d406f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.085293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-36f12dac-9eb3-4363-a85e-19dca75d406f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:30.086388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-36f12dac-9eb3-4363-a85e-19dca75d406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:30.095203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1045/2087] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:30.098953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:30.110416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27375148-c9e3-4842-bffd-7f0442525bbc service glance] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:30.116421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:30.117314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.117619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.117900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.118636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:30.118953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:30.120589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:30.133212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1043/2088] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:30.137517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f12dac-9eb3-4363-a85e-19dca75d406f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:30.142697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:30.150019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f12dac-9eb3-4363-a85e-19dca75d406f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:30.156265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615359d5-a610-4728-a351-6e6683ed2d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:30.157151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615359d5-a610-4728-a351-6e6683ed2d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.157616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615359d5-a610-4728-a351-6e6683ed2d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.158101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615359d5-a610-4728-a351-6e6683ed2d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.159051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615359d5-a610-4728-a351-6e6683ed2d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:30.161022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615359d5-a610-4728-a351-6e6683ed2d63 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:30.190086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1046/2089] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:30 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:30.194216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:30.207552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f5567-81d5-4ad1-bb34-72945416f991 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:30.248407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:30.277349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df3d09-88e0-4693-92c7-c2593d16aadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:30.278912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df3d09-88e0-4693-92c7-c2593d16aadf admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.279404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df3d09-88e0-4693-92c7-c2593d16aadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.279690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df3d09-88e0-4693-92c7-c2593d16aadf admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.280765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df3d09-88e0-4693-92c7-c2593d16aadf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:30.283441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df3d09-88e0-4693-92c7-c2593d16aadf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:30.308545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1044/2090] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:30 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:30.314022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615359d5-a610-4728-a351-6e6683ed2d63 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:30.334630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615359d5-a610-4728-a351-6e6683ed2d63 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:30.340817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ec2be-46ec-463e-b02d-f32f45a60bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:30.342067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec2be-46ec-463e-b02d-f32f45a60bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.342540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec2be-46ec-463e-b02d-f32f45a60bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.342998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec2be-46ec-463e-b02d-f32f45a60bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.344705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ec2be-46ec-463e-b02d-f32f45a60bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:30.347104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ec2be-46ec-463e-b02d-f32f45a60bb1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:30.360523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1047/2091] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:30.365516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df3d09-88e0-4693-92c7-c2593d16aadf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:30.381201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df3d09-88e0-4693-92c7-c2593d16aadf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:30.396011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b73815-63af-46ba-a1e0-6e45ac2e597b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:30.396011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b73815-63af-46ba-a1e0-6e45ac2e597b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.396011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b73815-63af-46ba-a1e0-6e45ac2e597b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.396011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b73815-63af-46ba-a1e0-6e45ac2e597b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.396364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b73815-63af-46ba-a1e0-6e45ac2e597b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:30.396364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b73815-63af-46ba-a1e0-6e45ac2e597b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:30.407379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1045/2092] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:30.412029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ec2be-46ec-463e-b02d-f32f45a60bb1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:30.424259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ec2be-46ec-463e-b02d-f32f45a60bb1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:30.435186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:30.436562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.438296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.438581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] PATH_INFO: `/v3/projects/eae1bcd894ca400ea2eb04926906edaa/users/f12943fa264049fba93a72026f2477ca/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.472931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae1bcd894ca400ea2eb04926906edaa, user_id=f12943fa264049fba93a72026f2477ca, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:30.476695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:30.506603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1048/2093] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:30 2022] PUT /identity/v3/projects/eae1bcd894ca400ea2eb04926906edaa/users/f12943fa264049fba93a72026f2477ca/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:30.519479 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916d329-a0f2-4911-a596-ba3c7d5652e0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.520623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916d329-a0f2-4911-a596-ba3c7d5652e0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.520769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916d329-a0f2-4911-a596-ba3c7d5652e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.597711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b916d329-a0f2-4911-a596-ba3c7d5652e0 None None] MFA Rules not processed for user `f12943fa264049fba93a72026f2477ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:30.603002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b916d329-a0f2-4911-a596-ba3c7d5652e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:30.750483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1046/2094] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:30.764063 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:30.766360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:30.864303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe6802-196c-4d12-9952-2a3195662684 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:30.869775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce63785-30c7-43a1-bdd3-8f84e79b04f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:30.870418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce63785-30c7-43a1-bdd3-8f84e79b04f7 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:30.870701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce63785-30c7-43a1-bdd3-8f84e79b04f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:30.870978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce63785-30c7-43a1-bdd3-8f84e79b04f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:30.881383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce63785-30c7-43a1-bdd3-8f84e79b04f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:30.883104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce63785-30c7-43a1-bdd3-8f84e79b04f7 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:30.894558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1049/2095] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:31.927213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916d329-a0f2-4911-a596-ba3c7d5652e0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:31.928693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b916d329-a0f2-4911-a596-ba3c7d5652e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:32.003864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916d329-a0f2-4911-a596-ba3c7d5652e0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:32.008739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:32.009623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:32.009911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:32.010190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] PATH_INFO: `/v3/users/f01e1d0fb7f34a509d5dcc1473a07d20` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:32.011453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01e1d0fb7f34a509d5dcc1473a07d20)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:32.013443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:32.059845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01e1d0fb7f34a509d5dcc1473a07d20'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:32.060983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:32.077513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01e1d0fb7f34a509d5dcc1473a07d20'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:32.078301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01e1d0fb7f34a509d5dcc1473a07d20'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:32.086338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01e1d0fb7f34a509d5dcc1473a07d20'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:32.087013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01e1d0fb7f34a509d5dcc1473a07d20'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:32.100243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1047/2096] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:31 2022] DELETE /identity/v3/users/f01e1d0fb7f34a509d5dcc1473a07d20 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:32.105396 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fe0d-188d-4b7f-bec5-39a5c9e101f4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:32.105935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fe0d-188d-4b7f-bec5-39a5c9e101f4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:32.106035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9fe0d-188d-4b7f-bec5-39a5c9e101f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:32.123661 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-72c9fe0d-188d-4b7f-bec5-39a5c9e101f4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:32.124943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-72c9fe0d-188d-4b7f-bec5-39a5c9e101f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:32.187871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1050/2097] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:32.197680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:32.199076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:32.259855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e510f-92e7-446c-8276-76c303e78d5d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:32.263530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be17dc-1d23-4b8c-9a81-0a7847eb50f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:32.264264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be17dc-1d23-4b8c-9a81-0a7847eb50f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:32.264565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be17dc-1d23-4b8c-9a81-0a7847eb50f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:32.264835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be17dc-1d23-4b8c-9a81-0a7847eb50f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:32.274098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be17dc-1d23-4b8c-9a81-0a7847eb50f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:32.275529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be17dc-1d23-4b8c-9a81-0a7847eb50f4 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:32.285505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1048/2098] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:32.895661 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9fe0d-188d-4b7f-bec5-39a5c9e101f4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:32.898946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-72c9fe0d-188d-4b7f-bec5-39a5c9e101f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:32.940359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9fe0d-188d-4b7f-bec5-39a5c9e101f4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:32.946819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:32.947812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:32.948187 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:32.948550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] PATH_INFO: `/v3/projects/dac5c0d03225472cbfbc93b02b55ca8c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:32.950092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac5c0d03225472cbfbc93b02b55ca8c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:32.952462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:32.987563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:33.009571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac5c0d03225472cbfbc93b02b55ca8c', 'request_id': 'req-40fa09cb-e080-48ad-8eaf-e88710aca206'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.019560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac5c0d03225472cbfbc93b02b55ca8c', 'request_id': 'req-40fa09cb-e080-48ad-8eaf-e88710aca206'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.026565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac5c0d03225472cbfbc93b02b55ca8c', 'request_id': 'req-40fa09cb-e080-48ad-8eaf-e88710aca206'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.038214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] The token cache is being invalidate because project dac5c0d03225472cbfbc93b02b55ca8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:33.038214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.038626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.040195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1051/2099] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:32 2022] DELETE /identity/v3/projects/dac5c0d03225472cbfbc93b02b55ca8c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:33.044295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be17dc-1d23-4b8c-9a81-0a7847eb50f4 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:33.055702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be17dc-1d23-4b8c-9a81-0a7847eb50f4 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:33.117367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:33.118241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:33.118521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:33.118797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] PATH_INFO: `/v3/users/442c00f7bef049bc94e6fa2c7ee9d16a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:33.119994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=442c00f7bef049bc94e6fa2c7ee9d16a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:33.121961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:33.178940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '442c00f7bef049bc94e6fa2c7ee9d16a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.192641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '442c00f7bef049bc94e6fa2c7ee9d16a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.193355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '442c00f7bef049bc94e6fa2c7ee9d16a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.197863 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:33.200759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '442c00f7bef049bc94e6fa2c7ee9d16a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.202258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '442c00f7bef049bc94e6fa2c7ee9d16a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.219302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1049/2100] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:33 2022] DELETE /identity/v3/users/442c00f7bef049bc94e6fa2c7ee9d16a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:33.506245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:33.507845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:33.526383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa09cb-e080-48ad-8eaf-e88710aca206 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:33.589860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:33.590717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:33.591090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:33.591305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] PATH_INFO: `/v3/projects/41a6ff3d1bb74eceadb43668704d8609` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:33.592542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a6ff3d1bb74eceadb43668704d8609)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:33.594666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:33.643331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a6ff3d1bb74eceadb43668704d8609', 'request_id': 'req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.651265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a6ff3d1bb74eceadb43668704d8609', 'request_id': 'req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.659074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a6ff3d1bb74eceadb43668704d8609', 'request_id': 'req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.669553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] The token cache is being invalidate because project 41a6ff3d1bb74eceadb43668704d8609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:33.669777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.670415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.671970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1052/2101] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:33 2022] DELETE /identity/v3/projects/41a6ff3d1bb74eceadb43668704d8609 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:33.676205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:33.677782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:33.692486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cf89e-95df-4025-8c0e-cfe2bc94acc8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:33.753494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:33.754987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:33.758951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:33.759832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:33.760113 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:33.760427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] PATH_INFO: `/v3/users/c26ed397260a4b1d9089f102034d8435` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:33.761619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26ed397260a4b1d9089f102034d8435)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:33.763544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:33.766896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c98bd2-9dc1-490e-a6c2-58a8cad99e58 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:33.771934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:33.772828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:33.773114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:33.773391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] PATH_INFO: `/v3/users/f12943fa264049fba93a72026f2477ca` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:33.774606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12943fa264049fba93a72026f2477ca)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:33.776549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:33.829151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26ed397260a4b1d9089f102034d8435'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.836368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:33.845705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12943fa264049fba93a72026f2477ca'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.846204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26ed397260a4b1d9089f102034d8435'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.846931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26ed397260a4b1d9089f102034d8435'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.854859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26ed397260a4b1d9089f102034d8435'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.855535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-63a47e18-e947-453b-bffc-36ef0874dd45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26ed397260a4b1d9089f102034d8435'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.862183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12943fa264049fba93a72026f2477ca'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.862918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12943fa264049fba93a72026f2477ca'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.863548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12943fa264049fba93a72026f2477ca'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.870861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12943fa264049fba93a72026f2477ca'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:33.877720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1050/2102] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:33 2022] DELETE /identity/v3/users/c26ed397260a4b1d9089f102034d8435 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:33.887842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1053/2103] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:33 2022] DELETE /identity/v3/users/f12943fa264049fba93a72026f2477ca => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:33.892664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6019ae-edd3-4737-b0e0-3bacebb88b84 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:33.892940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6019ae-edd3-4737-b0e0-3bacebb88b84 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:33.893227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6019ae-edd3-4737-b0e0-3bacebb88b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:33.912239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dc6019ae-edd3-4737-b0e0-3bacebb88b84 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:33.913468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dc6019ae-edd3-4737-b0e0-3bacebb88b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:33.983078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1051/2104] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:33.994034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:33.995445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:34.072770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253dbe42-2e3e-408c-bce0-19278ca90057 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:34.082489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a910353-9f0d-4a7d-a6e3-ee4bfe2a7d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:34.083423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a910353-9f0d-4a7d-a6e3-ee4bfe2a7d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:34.083764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a910353-9f0d-4a7d-a6e3-ee4bfe2a7d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:34.084317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a910353-9f0d-4a7d-a6e3-ee4bfe2a7d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:34.097844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a910353-9f0d-4a7d-a6e3-ee4bfe2a7d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:34.099333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a910353-9f0d-4a7d-a6e3-ee4bfe2a7d3e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:34.109474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1054/2105] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:34.606957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6019ae-edd3-4737-b0e0-3bacebb88b84 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:34.608527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dc6019ae-edd3-4737-b0e0-3bacebb88b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:34.624060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6019ae-edd3-4737-b0e0-3bacebb88b84 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:34.631678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:34.632644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:34.632933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:34.633243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] PATH_INFO: `/v3/projects/a047765440d14856a5887a392ab8f991` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:34.634490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a047765440d14856a5887a392ab8f991)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:34.636474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:34.638212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:34.723482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a047765440d14856a5887a392ab8f991', 'request_id': 'req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:34.730120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a047765440d14856a5887a392ab8f991', 'request_id': 'req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:34.738179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a047765440d14856a5887a392ab8f991', 'request_id': 'req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:34.753050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] The token cache is being invalidate because project a047765440d14856a5887a392ab8f991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:34.753405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:34.754211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:34.755868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1052/2106] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:34 2022] DELETE /identity/v3/projects/a047765440d14856a5887a392ab8f991 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:34.760826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b387e-d33d-45cf-abec-eb73080c27c5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:34.761329 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b387e-d33d-45cf-abec-eb73080c27c5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:34.761489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b387e-d33d-45cf-abec-eb73080c27c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:34.785311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-572b387e-d33d-45cf-abec-eb73080c27c5 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:34.786619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-572b387e-d33d-45cf-abec-eb73080c27c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:34.854235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:34.856050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:34.872212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1055/2107] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:34.950050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6fbae-accd-42be-99d8-dae735957402 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:34.950339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6fbae-accd-42be-99d8-dae735957402 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:34.950629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6fbae-accd-42be-99d8-dae735957402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:34.964793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e6e7b-6f58-4e05-b8cc-1259fd30711b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:34.970210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:34.971427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:34.971785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:34.972135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] PATH_INFO: `/v3/projects/eae1bcd894ca400ea2eb04926906edaa` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:34.973491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae1bcd894ca400ea2eb04926906edaa)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:34.975175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6fa6fbae-accd-42be-99d8-dae735957402 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:34.976293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:34.976413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6fa6fbae-accd-42be-99d8-dae735957402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:34.988200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1056/2108] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:34.989730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6fbae-accd-42be-99d8-dae735957402 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:34.999263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6fbae-accd-42be-99d8-dae735957402 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.004720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.005503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.006336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.006336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.007322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:35.007723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.009509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.023369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1057/2109] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:35.027672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.038772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdb590-030d-4038-bbee-62fb3614a1d6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.053431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae1bcd894ca400ea2eb04926906edaa', 'request_id': 'req-f302f93d-839e-483a-a168-1dcfd9311d5b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:35.060067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae1bcd894ca400ea2eb04926906edaa', 'request_id': 'req-f302f93d-839e-483a-a168-1dcfd9311d5b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:35.066710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae1bcd894ca400ea2eb04926906edaa', 'request_id': 'req-f302f93d-839e-483a-a168-1dcfd9311d5b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:35.084782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] The token cache is being invalidate because project eae1bcd894ca400ea2eb04926906edaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:35.085072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:35.085703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f302f93d-839e-483a-a168-1dcfd9311d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:35.087284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1053/2110] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:34 2022] DELETE /identity/v3/projects/eae1bcd894ca400ea2eb04926906edaa => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:35.088799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:35.092673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded97c6-7fc3-4165-9d9f-92fe81dbd08d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.092884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded97c6-7fc3-4165-9d9f-92fe81dbd08d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.093215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded97c6-7fc3-4165-9d9f-92fe81dbd08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.114620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1ded97c6-7fc3-4165-9d9f-92fe81dbd08d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:35.115860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1ded97c6-7fc3-4165-9d9f-92fe81dbd08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:35.116620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.117300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.117586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.117865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.118568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.120561 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.124724 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1054/2111] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:35.132820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcc3d3-a4c0-4b9c-b3ed-21bac342175a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.133105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcc3d3-a4c0-4b9c-b3ed-21bac342175a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.133378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcc3d3-a4c0-4b9c-b3ed-21bac342175a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.157037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1058/2112] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:35 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:35.159802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67fcc3d3-a4c0-4b9c-b3ed-21bac342175a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:35.160974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.161210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67fcc3d3-a4c0-4b9c-b3ed-21bac342175a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:35.164659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:35.174755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:35.239604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1055/2113] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:35.241601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf06415-42f4-4533-b742-d4570eae8c7b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.246401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcc3d3-a4c0-4b9c-b3ed-21bac342175a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.247297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8493fbe-4302-40bf-9715-ab2f777a7f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.247987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8493fbe-4302-40bf-9715-ab2f777a7f6b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.248422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8493fbe-4302-40bf-9715-ab2f777a7f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.248707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8493fbe-4302-40bf-9715-ab2f777a7f6b admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.249412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8493fbe-4302-40bf-9715-ab2f777a7f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.251954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8493fbe-4302-40bf-9715-ab2f777a7f6b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.256683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcc3d3-a4c0-4b9c-b3ed-21bac342175a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.262297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.262979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.263300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.263573 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.264364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:35.264692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.266987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.277555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1059/2114] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:35 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:35.278877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8493fbe-4302-40bf-9715-ab2f777a7f6b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.283991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1056/2115] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 12:05:35.285329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.290125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8493fbe-4302-40bf-9715-ab2f777a7f6b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.295972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c48e5-8ff9-4bec-b6b8-34767ca4a513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.296361 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b5efb-8e32-4d33-a768-1583aa92addd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.296916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c48e5-8ff9-4bec-b6b8-34767ca4a513 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.297877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c48e5-8ff9-4bec-b6b8-34767ca4a513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.298198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c48e5-8ff9-4bec-b6b8-34767ca4a513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.299220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c48e5-8ff9-4bec-b6b8-34767ca4a513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.300976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c48e5-8ff9-4bec-b6b8-34767ca4a513 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.303009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47f033-0cdf-4b1c-9285-1afb8c3dfb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.303681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f033-0cdf-4b1c-9285-1afb8c3dfb3a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.303973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f033-0cdf-4b1c-9285-1afb8c3dfb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.304381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47f033-0cdf-4b1c-9285-1afb8c3dfb3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.305208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47f033-0cdf-4b1c-9285-1afb8c3dfb3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.307191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47f033-0cdf-4b1c-9285-1afb8c3dfb3a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.315921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1060/2116] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:35.319602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c48e5-8ff9-4bec-b6b8-34767ca4a513 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.331730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c48e5-8ff9-4bec-b6b8-34767ca4a513 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.336717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4fbe3-d26d-463c-bb1f-191fbcb0ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.337402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4fbe3-d26d-463c-bb1f-191fbcb0ee7f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.337690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4fbe3-d26d-463c-bb1f-191fbcb0ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.337978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4fbe3-d26d-463c-bb1f-191fbcb0ee7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.338558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1057/2117] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:35 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:35.342317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4fbe3-d26d-463c-bb1f-191fbcb0ee7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.343285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47f033-0cdf-4b1c-9285-1afb8c3dfb3a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.344385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4fbe3-d26d-463c-bb1f-191fbcb0ee7f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.358558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1061/2118] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:35.360259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4fbe3-d26d-463c-bb1f-191fbcb0ee7f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.372372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4fbe3-d26d-463c-bb1f-191fbcb0ee7f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.412994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47f033-0cdf-4b1c-9285-1afb8c3dfb3a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.418322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d78259-cf5b-47d0-9e50-86e56e70f07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.419004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d78259-cf5b-47d0-9e50-86e56e70f07d admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.419288 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d78259-cf5b-47d0-9e50-86e56e70f07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.419568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d78259-cf5b-47d0-9e50-86e56e70f07d admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.420294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d78259-cf5b-47d0-9e50-86e56e70f07d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.422386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d78259-cf5b-47d0-9e50-86e56e70f07d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.442722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f7301-3d1e-4653-b638-21d462bdbd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.443632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f7301-3d1e-4653-b638-21d462bdbd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.443922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f7301-3d1e-4653-b638-21d462bdbd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.444238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f7301-3d1e-4653-b638-21d462bdbd6f admin admin] PATH_INFO: `/v3/projects/701bba02c995487f81e17bd916730396/users/317b7259e0344f238a06da0521ee348a/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.446050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1058/2119] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:35 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:35.450915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d78259-cf5b-47d0-9e50-86e56e70f07d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.462247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f7301-3d1e-4653-b638-21d462bdbd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=701bba02c995487f81e17bd916730396, user_id=317b7259e0344f238a06da0521ee348a, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.462997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d78259-cf5b-47d0-9e50-86e56e70f07d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.465000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f7301-3d1e-4653-b638-21d462bdbd6f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.467756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8b2f0-310c-4185-8249-71ea666a4a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.468595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8b2f0-310c-4185-8249-71ea666a4a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.468881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8b2f0-310c-4185-8249-71ea666a4a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.469191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8b2f0-310c-4185-8249-71ea666a4a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.469917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8b2f0-310c-4185-8249-71ea666a4a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.471511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8b2f0-310c-4185-8249-71ea666a4a2e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.479928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1062/2120] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:35 2022] PUT /identity/v3/projects/701bba02c995487f81e17bd916730396/users/317b7259e0344f238a06da0521ee348a/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:35.484182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1059/2121] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:05:35.486378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f7301-3d1e-4653-b638-21d462bdbd6f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.491074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b466c-8f2e-4427-bd81-9d11d542bd52 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.491362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b466c-8f2e-4427-bd81-9d11d542bd52 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.491637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b466c-8f2e-4427-bd81-9d11d542bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.516912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:35.528998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7a3b466c-8f2e-4427-bd81-9d11d542bd52 None None] MFA Rules not processed for user `317b7259e0344f238a06da0521ee348a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:35.530179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a3b466c-8f2e-4427-bd81-9d11d542bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:35.580214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f7301-3d1e-4653-b638-21d462bdbd6f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.585548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d0e15-841c-4b8f-b309-f8bd1d8b2210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.586236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d0e15-841c-4b8f-b309-f8bd1d8b2210 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.586522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d0e15-841c-4b8f-b309-f8bd1d8b2210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.590818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d0e15-841c-4b8f-b309-f8bd1d8b2210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.590818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d0e15-841c-4b8f-b309-f8bd1d8b2210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.590818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d0e15-841c-4b8f-b309-f8bd1d8b2210 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.595866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:35.609012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1063/2122] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:35.612717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d0e15-841c-4b8f-b309-f8bd1d8b2210 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.624591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d0e15-841c-4b8f-b309-f8bd1d8b2210 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.630895 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f2a77-ed8d-4fc5-8be7-735bf65693fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.631805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f2a77-ed8d-4fc5-8be7-735bf65693fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.632089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f2a77-ed8d-4fc5-8be7-735bf65693fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.632395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f2a77-ed8d-4fc5-8be7-735bf65693fd admin admin] PATH_INFO: `/v3/projects/d1a1a2c48b364bf9b84c85a0f1ca15fd/users/5ea25f84a83f419dbc524d8debd56a9f/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.638867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1060/2123] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:35.646106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b466c-8f2e-4427-bd81-9d11d542bd52 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:35.647653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a3b466c-8f2e-4427-bd81-9d11d542bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:35.653550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f2a77-ed8d-4fc5-8be7-735bf65693fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a1a2c48b364bf9b84c85a0f1ca15fd, user_id=5ea25f84a83f419dbc524d8debd56a9f, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.657107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f2a77-ed8d-4fc5-8be7-735bf65693fd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.683319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1064/2124] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:35 2022] PUT /identity/v3/projects/d1a1a2c48b364bf9b84c85a0f1ca15fd/users/5ea25f84a83f419dbc524d8debd56a9f/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:35.695238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4b4b4-f74f-4835-9c34-d113395f1a6d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.695529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4b4b4-f74f-4835-9c34-d113395f1a6d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.696198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4b4b4-f74f-4835-9c34-d113395f1a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.728282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4fd4b4b4-f74f-4835-9c34-d113395f1a6d None None] MFA Rules not processed for user `5ea25f84a83f419dbc524d8debd56a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:35.729524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4fd4b4b4-f74f-4835-9c34-d113395f1a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:35.734401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b466c-8f2e-4427-bd81-9d11d542bd52 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:35.805300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94301b7c-643d-498f-820d-bbfcb06d9a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:35.806452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301b7c-643d-498f-820d-bbfcb06d9a28 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:35.806747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301b7c-643d-498f-820d-bbfcb06d9a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:35.807025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301b7c-643d-498f-820d-bbfcb06d9a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:35.811195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1065/2125] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:35.879978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94301b7c-643d-498f-820d-bbfcb06d9a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:35.881639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94301b7c-643d-498f-820d-bbfcb06d9a28 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:35.893004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1061/2126] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:36.018085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4b4b4-f74f-4835-9c34-d113395f1a6d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:36.019926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4fd4b4b4-f74f-4835-9c34-d113395f1a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:36.032749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4b4b4-f74f-4835-9c34-d113395f1a6d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:36.037374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dc4b4-0cdd-49f8-a64c-eb74936cebca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:36.038065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dc4b4-0cdd-49f8-a64c-eb74936cebca service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:36.038347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dc4b4-0cdd-49f8-a64c-eb74936cebca service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:36.039197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dc4b4-0cdd-49f8-a64c-eb74936cebca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:36.049711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dc4b4-0cdd-49f8-a64c-eb74936cebca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:36.051558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dc4b4-0cdd-49f8-a64c-eb74936cebca service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:36.063267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1066/2127] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:41.969228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d692e9-0393-4f7f-adcd-4b9287d32279 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:41.969858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d692e9-0393-4f7f-adcd-4b9287d32279 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:41.969858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d692e9-0393-4f7f-adcd-4b9287d32279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:41.987950 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-57d692e9-0393-4f7f-adcd-4b9287d32279 None None] MFA Rules not processed for user `a440b40bc3124bafa502f66a004d0de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:41.989216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-57d692e9-0393-4f7f-adcd-4b9287d32279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:42.050902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1062/2128] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:46.057535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=70,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:47.875822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dc4b4-0cdd-49f8-a64c-eb74936cebca service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:47.942294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dc4b4-0cdd-49f8-a64c-eb74936cebca service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:47.947155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:47.948035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:47.948355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:47.948695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] PATH_INFO: `/v3/users/5ea25f84a83f419dbc524d8debd56a9f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:47.949902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea25f84a83f419dbc524d8debd56a9f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:47.951912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:48.005415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea25f84a83f419dbc524d8debd56a9f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.022878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea25f84a83f419dbc524d8debd56a9f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.023855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea25f84a83f419dbc524d8debd56a9f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.024666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea25f84a83f419dbc524d8debd56a9f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.032953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea25f84a83f419dbc524d8debd56a9f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.048411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1067/2129] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:05:47 2022] DELETE /identity/v3/users/5ea25f84a83f419dbc524d8debd56a9f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:48.052958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb257-f30e-4961-8d25-0ff6e05e1075 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:48.053520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb257-f30e-4961-8d25-0ff6e05e1075 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:48.053520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deb257-f30e-4961-8d25-0ff6e05e1075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:48.059191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:48.073202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-09deb257-f30e-4961-8d25-0ff6e05e1075 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:48.074438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-09deb257-f30e-4961-8d25-0ff6e05e1075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:48.141864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1063/2130] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:05:48.151744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:48.153228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:48.220458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4395417-f2cb-4a3c-8c8d-ccebf64dcd6c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:48.225270 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe13b2d-3699-4fc1-9bfd-9269c96037c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:48.226014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13b2d-3699-4fc1-9bfd-9269c96037c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:48.226292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13b2d-3699-4fc1-9bfd-9269c96037c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:48.226574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe13b2d-3699-4fc1-9bfd-9269c96037c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:48.238362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe13b2d-3699-4fc1-9bfd-9269c96037c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:48.239850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe13b2d-3699-4fc1-9bfd-9269c96037c6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:48.251087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1068/2131] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:48.693600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deb257-f30e-4961-8d25-0ff6e05e1075 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:48.694853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-09deb257-f30e-4961-8d25-0ff6e05e1075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:48.707818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deb257-f30e-4961-8d25-0ff6e05e1075 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:48.713167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:48.714109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:48.714397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:48.714666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] PATH_INFO: `/v3/projects/d1a1a2c48b364bf9b84c85a0f1ca15fd` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:48.715904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a1a2c48b364bf9b84c85a0f1ca15fd)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:48.717907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:48.772222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a1a2c48b364bf9b84c85a0f1ca15fd', 'request_id': 'req-41f85732-77fd-4a17-bddf-dba39a8e2e08'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.778997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a1a2c48b364bf9b84c85a0f1ca15fd', 'request_id': 'req-41f85732-77fd-4a17-bddf-dba39a8e2e08'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.786284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a1a2c48b364bf9b84c85a0f1ca15fd', 'request_id': 'req-41f85732-77fd-4a17-bddf-dba39a8e2e08'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.803689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] The token cache is being invalidate because project d1a1a2c48b364bf9b84c85a0f1ca15fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:05:48.803954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.804689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-41f85732-77fd-4a17-bddf-dba39a8e2e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:05:48.809835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1064/2132] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:05:48 2022] DELETE /identity/v3/projects/d1a1a2c48b364bf9b84c85a0f1ca15fd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:48.818076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b55610-1b25-47f9-a5ff-e2747f490c5d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:48.819117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b55610-1b25-47f9-a5ff-e2747f490c5d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:48.819409 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b55610-1b25-47f9-a5ff-e2747f490c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:48.842917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-23b55610-1b25-47f9-a5ff-e2747f490c5d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:48.845098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-23b55610-1b25-47f9-a5ff-e2747f490c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:48.863340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:48.928499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1069/2133] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:48.932459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1925f-e0eb-4da7-acc2-aa17da85f2b1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:48.933095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1925f-e0eb-4da7-acc2-aa17da85f2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:48.933529 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1925f-e0eb-4da7-acc2-aa17da85f2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:48.967697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bde1925f-e0eb-4da7-acc2-aa17da85f2b1 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:48.969550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bde1925f-e0eb-4da7-acc2-aa17da85f2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:48.977677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1065/2134] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:48.981987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b55610-1b25-47f9-a5ff-e2747f490c5d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:48.992896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b55610-1b25-47f9-a5ff-e2747f490c5d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:48.998212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:48.998897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:48.999190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:48.999517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.000404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:05:49.000845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.002739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.015898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1070/2135] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:05:49.020644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1925f-e0eb-4da7-acc2-aa17da85f2b1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.029871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1925f-e0eb-4da7-acc2-aa17da85f2b1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.036710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8a42f-54cd-4935-b27b-678bec1ecc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.037393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a42f-54cd-4935-b27b-678bec1ecc6a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.037676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a42f-54cd-4935-b27b-678bec1ecc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.037953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a42f-54cd-4935-b27b-678bec1ecc6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.038641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8a42f-54cd-4935-b27b-678bec1ecc6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.040581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8a42f-54cd-4935-b27b-678bec1ecc6a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.069039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1066/2136] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:49 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:49.073226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.084088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffb2b8-5d5a-4ab6-8e5b-1dec4acef141 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.140110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5bbb-e583-4556-a924-fdc38c9821a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.141310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5bbb-e583-4556-a924-fdc38c9821a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.141590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5bbb-e583-4556-a924-fdc38c9821a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.141875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b5bbb-e583-4556-a924-fdc38c9821a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.142563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b5bbb-e583-4556-a924-fdc38c9821a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.144541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b5bbb-e583-4556-a924-fdc38c9821a2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.165254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1071/2137] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:49 2022] POST /identity/v3/users => generated 279 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:49.169372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8a42f-54cd-4935-b27b-678bec1ecc6a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.180209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8a42f-54cd-4935-b27b-678bec1ecc6a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.185328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c079568-78b0-4ab6-9b61-5bee405699b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.185991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c079568-78b0-4ab6-9b61-5bee405699b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.186280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c079568-78b0-4ab6-9b61-5bee405699b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.186558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c079568-78b0-4ab6-9b61-5bee405699b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.187697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c079568-78b0-4ab6-9b61-5bee405699b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.190372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c079568-78b0-4ab6-9b61-5bee405699b9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.201459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1067/2138] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:49.206183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5bbb-e583-4556-a924-fdc38c9821a2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.216463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b5bbb-e583-4556-a924-fdc38c9821a2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.222191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c6bb5-70bd-43c9-89e5-92eda4cc8e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.222841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c6bb5-70bd-43c9-89e5-92eda4cc8e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.223130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c6bb5-70bd-43c9-89e5-92eda4cc8e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.223412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c6bb5-70bd-43c9-89e5-92eda4cc8e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.224140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c6bb5-70bd-43c9-89e5-92eda4cc8e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.225987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c6bb5-70bd-43c9-89e5-92eda4cc8e76 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.236917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1072/2139] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:49.241254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c079568-78b0-4ab6-9b61-5bee405699b9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.251504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c079568-78b0-4ab6-9b61-5bee405699b9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.256370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca293c4-71d5-436a-ac8f-ca9f983b680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.257291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca293c4-71d5-436a-ac8f-ca9f983b680f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.257669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca293c4-71d5-436a-ac8f-ca9f983b680f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.258143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca293c4-71d5-436a-ac8f-ca9f983b680f admin admin] PATH_INFO: `/v3/projects/0634338dc25f45929a4ce9871d7ac747/users/4fdc660b9eec4b6ba6d69761176d61db/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.283081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca293c4-71d5-436a-ac8f-ca9f983b680f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0634338dc25f45929a4ce9871d7ac747, user_id=4fdc660b9eec4b6ba6d69761176d61db, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.286510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca293c4-71d5-436a-ac8f-ca9f983b680f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.301801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1068/2140] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:49 2022] PUT /identity/v3/projects/0634338dc25f45929a4ce9871d7ac747/users/4fdc660b9eec4b6ba6d69761176d61db/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:05:49.312792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b4c6f-87f3-456a-9d2f-cc0e3b77bb1f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.313084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b4c6f-87f3-456a-9d2f-cc0e3b77bb1f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.313386 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b4c6f-87f3-456a-9d2f-cc0e3b77bb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.341193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-af0b4c6f-87f3-456a-9d2f-cc0e3b77bb1f None None] MFA Rules not processed for user `4fdc660b9eec4b6ba6d69761176d61db`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:49.342577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af0b4c6f-87f3-456a-9d2f-cc0e3b77bb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:49.409616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1073/2141] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:49.414827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca293c4-71d5-436a-ac8f-ca9f983b680f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.442775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:49.481416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca293c4-71d5-436a-ac8f-ca9f983b680f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.486330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df8357-5e21-4391-9443-b690a5d367b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.486992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df8357-5e21-4391-9443-b690a5d367b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.487277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df8357-5e21-4391-9443-b690a5d367b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.487567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df8357-5e21-4391-9443-b690a5d367b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.488285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df8357-5e21-4391-9443-b690a5d367b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.490237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df8357-5e21-4391-9443-b690a5d367b6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.515957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1069/2142] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:05:49 2022] POST /identity/v3/projects => generated 357 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:05:49.520531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b4c6f-87f3-456a-9d2f-cc0e3b77bb1f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.531428 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b4c6f-87f3-456a-9d2f-cc0e3b77bb1f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.544192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:49.598630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ada52c-367a-4c83-b58d-cfcc29468d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.599339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ada52c-367a-4c83-b58d-cfcc29468d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.599624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ada52c-367a-4c83-b58d-cfcc29468d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.599897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ada52c-367a-4c83-b58d-cfcc29468d81 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.600634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ada52c-367a-4c83-b58d-cfcc29468d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.602584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ada52c-367a-4c83-b58d-cfcc29468d81 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.623351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1074/2143] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:05:49 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:05:49.628024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df8357-5e21-4391-9443-b690a5d367b6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.639410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df8357-5e21-4391-9443-b690a5d367b6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.645299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd24a11-1339-4846-b2ce-0c7020ac3add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.645972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd24a11-1339-4846-b2ce-0c7020ac3add admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.646250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd24a11-1339-4846-b2ce-0c7020ac3add admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.646532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd24a11-1339-4846-b2ce-0c7020ac3add admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.647256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd24a11-1339-4846-b2ce-0c7020ac3add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.648889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd24a11-1339-4846-b2ce-0c7020ac3add admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.659426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1070/2144] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:49.663766 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ada52c-367a-4c83-b58d-cfcc29468d81 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.675256 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ada52c-367a-4c83-b58d-cfcc29468d81 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.680080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e27a-09c5-4f18-b750-d8b0f8f58a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.681114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e27a-09c5-4f18-b750-d8b0f8f58a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.681493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e27a-09c5-4f18-b750-d8b0f8f58a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.681848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e27a-09c5-4f18-b750-d8b0f8f58a53 admin admin] PATH_INFO: `/v3/projects/70ba8039032d4e0fa9dd06b1a6b344e7/users/afaa42a0adff442cad7c15b7b452b837/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.697693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58e27a-09c5-4f18-b750-d8b0f8f58a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70ba8039032d4e0fa9dd06b1a6b344e7, user_id=afaa42a0adff442cad7c15b7b452b837, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.700416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58e27a-09c5-4f18-b750-d8b0f8f58a53 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.715196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1075/2145] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:49 2022] PUT /identity/v3/projects/70ba8039032d4e0fa9dd06b1a6b344e7/users/afaa42a0adff442cad7c15b7b452b837/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:49.719521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd24a11-1339-4846-b2ce-0c7020ac3add admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.729892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd24a11-1339-4846-b2ce-0c7020ac3add admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.786375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e43164-afab-4b8c-94bf-f6833779e1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.787053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e43164-afab-4b8c-94bf-f6833779e1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.787344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e43164-afab-4b8c-94bf-f6833779e1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.787626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e43164-afab-4b8c-94bf-f6833779e1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.788385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e43164-afab-4b8c-94bf-f6833779e1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.790005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e43164-afab-4b8c-94bf-f6833779e1c9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.800466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1071/2146] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:05:49.804587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e27a-09c5-4f18-b750-d8b0f8f58a53 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.815459 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e27a-09c5-4f18-b750-d8b0f8f58a53 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:49.820320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:49.821198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.821478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.821748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] PATH_INFO: `/v3/projects/70ba8039032d4e0fa9dd06b1a6b344e7/users/afaa42a0adff442cad7c15b7b452b837/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.823700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70ba8039032d4e0fa9dd06b1a6b344e7, user_id=afaa42a0adff442cad7c15b7b452b837, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:49.826366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:49.837421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1076/2147] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:05:49 2022] PUT /identity/v3/projects/70ba8039032d4e0fa9dd06b1a6b344e7/users/afaa42a0adff442cad7c15b7b452b837/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:05:49.847738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a469a8e-beaf-43cb-a757-3068e9740e94 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:49.848038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a469a8e-beaf-43cb-a757-3068e9740e94 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:49.848355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a469a8e-beaf-43cb-a757-3068e9740e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:49.876874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8a469a8e-beaf-43cb-a757-3068e9740e94 None None] MFA Rules not processed for user `afaa42a0adff442cad7c15b7b452b837`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:05:49.878126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a469a8e-beaf-43cb-a757-3068e9740e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:49.952658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1072/2148] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:05:49.965103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:49.967271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:50.043771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d8666-bbed-4602-9489-e9fd6e479613 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:50.052875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:50.053548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:50.053899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:50.054116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:50.070610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:50.072095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:50.085150 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1077/2149] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:05:50.179462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a469a8e-beaf-43cb-a757-3068e9740e94 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:05:50.181133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a469a8e-beaf-43cb-a757-3068e9740e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:05:50.194625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a469a8e-beaf-43cb-a757-3068e9740e94 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:05:50.198447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c110f1-63cf-4120-9ef1-03086e03b9d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:05:50.199105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c110f1-63cf-4120-9ef1-03086e03b9d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:05:50.199390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c110f1-63cf-4120-9ef1-03086e03b9d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:05:50.199671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c110f1-63cf-4120-9ef1-03086e03b9d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:05:50.213909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:05:50.267503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c110f1-63cf-4120-9ef1-03086e03b9d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:05:50.269140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c110f1-63cf-4120-9ef1-03086e03b9d4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:05:50.278517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1073/2150] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:06:00.077900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:00.274178 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=20 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:09.442359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:09.443820 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:09.515095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9be82-194c-4001-9641-446e3c6139ef service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:09.521850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:09.522823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:09.523128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:09.523409 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] PATH_INFO: `/v3/users/2cf75e4a8c934c59aeac173052a84653` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:09.524746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf75e4a8c934c59aeac173052a84653)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:09.527029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:09.593658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf75e4a8c934c59aeac173052a84653'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:09.612273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf75e4a8c934c59aeac173052a84653'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:09.613348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf75e4a8c934c59aeac173052a84653'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:09.613979 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf75e4a8c934c59aeac173052a84653'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:09.621789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf75e4a8c934c59aeac173052a84653'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:09.635733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1078/2151] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:06:09 2022] DELETE /identity/v3/users/2cf75e4a8c934c59aeac173052a84653 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:09.640507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4644f6-c351-48c3-b3e9-8da3e3a5cd14 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:09.641347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4644f6-c351-48c3-b3e9-8da3e3a5cd14 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:09.641347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4644f6-c351-48c3-b3e9-8da3e3a5cd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:09.661311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4a4644f6-c351-48c3-b3e9-8da3e3a5cd14 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:09.662535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4a4644f6-c351-48c3-b3e9-8da3e3a5cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:09.726617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1074/2152] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:09.737117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:09.738598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:09.805560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6df298-e92a-4e51-b3e0-807a9db8ab1f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:09.809506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:09.810220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:09.810514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:09.810797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:09.820526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:09.822364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:09.833347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1079/2153] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:06:10.149280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4644f6-c351-48c3-b3e9-8da3e3a5cd14 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:10.151526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4a4644f6-c351-48c3-b3e9-8da3e3a5cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:10.291467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:10.292898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:10.314833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4644f6-c351-48c3-b3e9-8da3e3a5cd14 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:10.328710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:10.330325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.331160 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.333124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] PATH_INFO: `/v3/users/4fdc660b9eec4b6ba6d69761176d61db` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.336357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fdc660b9eec4b6ba6d69761176d61db)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:10.355730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:10.368670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:10.440382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fdc660b9eec4b6ba6d69761176d61db'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.441155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda584f7-0ba0-46ef-b979-e1a34682a360 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:10.448116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:10.449738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.450228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.450653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] PATH_INFO: `/v3/projects/eec44ec24e8f4194b4812dacf57bbfe9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.452003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec44ec24e8f4194b4812dacf57bbfe9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:10.455769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:10.457003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fdc660b9eec4b6ba6d69761176d61db'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.458114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fdc660b9eec4b6ba6d69761176d61db'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.476254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fdc660b9eec4b6ba6d69761176d61db'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.477212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-39bb8f31-c8f7-4939-95e4-ffd7e2fb3f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fdc660b9eec4b6ba6d69761176d61db'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.496037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1075/2154] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:06:10 2022] DELETE /identity/v3/users/4fdc660b9eec4b6ba6d69761176d61db => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:10.497755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579d643-62e3-4958-b204-0290a2262733 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.498754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579d643-62e3-4958-b204-0290a2262733 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.499034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579d643-62e3-4958-b204-0290a2262733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.523432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:10.533553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9579d643-62e3-4958-b204-0290a2262733 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:10.535855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9579d643-62e3-4958-b204-0290a2262733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:10.542625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec44ec24e8f4194b4812dacf57bbfe9', 'request_id': 'req-bc809574-205e-4102-addb-3bdaa2670d70'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.555620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec44ec24e8f4194b4812dacf57bbfe9', 'request_id': 'req-bc809574-205e-4102-addb-3bdaa2670d70'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.567178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec44ec24e8f4194b4812dacf57bbfe9', 'request_id': 'req-bc809574-205e-4102-addb-3bdaa2670d70'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.590105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] The token cache is being invalidate because project eec44ec24e8f4194b4812dacf57bbfe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:06:10.590707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.591713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bc809574-205e-4102-addb-3bdaa2670d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:10.594751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1080/2155] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:06:10 2022] DELETE /identity/v3/projects/eec44ec24e8f4194b4812dacf57bbfe9 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:10.605983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62280406-c0dc-478d-921a-60dfb1d47b1b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.606385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62280406-c0dc-478d-921a-60dfb1d47b1b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.606718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62280406-c0dc-478d-921a-60dfb1d47b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.641604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1076/2156] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:06:10.659210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579d643-62e3-4958-b204-0290a2262733 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:10.660931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9579d643-62e3-4958-b204-0290a2262733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:10.688193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-62280406-c0dc-478d-921a-60dfb1d47b1b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:10.689627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-62280406-c0dc-478d-921a-60dfb1d47b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:10.702211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1081/2157] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:10.710190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2054-65ef-4c90-b542-f01643376b12 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.710490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2054-65ef-4c90-b542-f01643376b12 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.710832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2054-65ef-4c90-b542-f01643376b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.738327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2bbf2054-65ef-4c90-b542-f01643376b12 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:10.739687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2bbf2054-65ef-4c90-b542-f01643376b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:10.755181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1082/2158] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:10.760771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf2054-65ef-4c90-b542-f01643376b12 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:10.772806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf2054-65ef-4c90-b542-f01643376b12 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:10.777951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:10.778576 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.778873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.779158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.779897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:06:10.780251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:10.782761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:10.801044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1083/2159] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:06:10.802098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579d643-62e3-4958-b204-0290a2262733 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:10.805904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44549ac2-8b82-4c87-80f6-1de0e5c8ce40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:10.806552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549ac2-8b82-4c87-80f6-1de0e5c8ce40 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.806838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549ac2-8b82-4c87-80f6-1de0e5c8ce40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.807116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549ac2-8b82-4c87-80f6-1de0e5c8ce40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.809626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:10.818211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44549ac2-8b82-4c87-80f6-1de0e5c8ce40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:10.819651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44549ac2-8b82-4c87-80f6-1de0e5c8ce40 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:10.820447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c1e2f-4e88-4647-8203-a7c275076c14 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:10.827167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4207-c4fb-4872-b14d-d388c24ba2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:10.827970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4207-c4fb-4872-b14d-d388c24ba2fd admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.828419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4207-c4fb-4872-b14d-d388c24ba2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.828805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4207-c4fb-4872-b14d-d388c24ba2fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.830118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4207-c4fb-4872-b14d-d388c24ba2fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:10.830567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1077/2160] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:06:10.834358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4207-c4fb-4872-b14d-d388c24ba2fd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:10.873106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1084/2161] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:06:10 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:06:10.878386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44549ac2-8b82-4c87-80f6-1de0e5c8ce40 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:10.889199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44549ac2-8b82-4c87-80f6-1de0e5c8ce40 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:10.986006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe4a27-3e66-4bce-9fa2-f14cb08b3d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:10.987341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe4a27-3e66-4bce-9fa2-f14cb08b3d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:10.987931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe4a27-3e66-4bce-9fa2-f14cb08b3d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:10.988601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe4a27-3e66-4bce-9fa2-f14cb08b3d7f admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:10.989604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe4a27-3e66-4bce-9fa2-f14cb08b3d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:10.992962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe4a27-3e66-4bce-9fa2-f14cb08b3d7f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:11.053286 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1078/2162] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:06:10 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:06:11.057167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4207-c4fb-4872-b14d-d388c24ba2fd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:11.073759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4207-c4fb-4872-b14d-d388c24ba2fd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:11.093540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c1a28-9cf3-4c66-abe7-f93d71d648c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:11.095083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c1a28-9cf3-4c66-abe7-f93d71d648c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:11.095378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c1a28-9cf3-4c66-abe7-f93d71d648c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:11.095661 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c1a28-9cf3-4c66-abe7-f93d71d648c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:11.096445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c1a28-9cf3-4c66-abe7-f93d71d648c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:11.098081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c1a28-9cf3-4c66-abe7-f93d71d648c7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:11.109131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1085/2163] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:06:11.113980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe4a27-3e66-4bce-9fa2-f14cb08b3d7f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:11.119402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:11.125188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe4a27-3e66-4bce-9fa2-f14cb08b3d7f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:11.135347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:11.135823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:11.136111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:11.136425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:11.137170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:11.138786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:11.154893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1079/2164] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:06:11.159918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c1a28-9cf3-4c66-abe7-f93d71d648c7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:11.175448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c1a28-9cf3-4c66-abe7-f93d71d648c7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:11.179424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:11.180391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f8698-a6ac-4674-9d66-44ce404c6ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:11.181184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f8698-a6ac-4674-9d66-44ce404c6ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:11.181470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f8698-a6ac-4674-9d66-44ce404c6ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:11.181749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f8698-a6ac-4674-9d66-44ce404c6ac5 admin admin] PATH_INFO: `/v3/projects/94cd1e0377594f52bb5bfa7159731d48/users/1d4c83a1ccfc43beb3f48e4462b06794/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:11.181844 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:11.196534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d52d7-7e9f-4c81-b8a3-199c22c4dc3f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:11.200617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f8698-a6ac-4674-9d66-44ce404c6ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cd1e0377594f52bb5bfa7159731d48, user_id=1d4c83a1ccfc43beb3f48e4462b06794, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:11.202258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:11.203530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f8698-a6ac-4674-9d66-44ce404c6ac5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:11.203982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:11.204272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:11.204540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] PATH_INFO: `/v3/projects/0634338dc25f45929a4ce9871d7ac747` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:11.206068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0634338dc25f45929a4ce9871d7ac747)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:11.207965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:11.235153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1086/2165] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:06:11 2022] PUT /identity/v3/projects/94cd1e0377594f52bb5bfa7159731d48/users/1d4c83a1ccfc43beb3f48e4462b06794/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:11.236729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c2f9-6d52-4b4c-a927-50acc7cb7040 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:11.237029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c2f9-6d52-4b4c-a927-50acc7cb7040 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:11.237315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c2f9-6d52-4b4c-a927-50acc7cb7040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:11.255488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fb67c2f9-6d52-4b4c-a927-50acc7cb7040 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:11.256742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fb67c2f9-6d52-4b4c-a927-50acc7cb7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:11.271413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0634338dc25f45929a4ce9871d7ac747', 'request_id': 'req-aaed274f-2736-4cdd-b6a6-06781ebafb83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.276341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:11.279758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0634338dc25f45929a4ce9871d7ac747', 'request_id': 'req-aaed274f-2736-4cdd-b6a6-06781ebafb83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.286954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0634338dc25f45929a4ce9871d7ac747', 'request_id': 'req-aaed274f-2736-4cdd-b6a6-06781ebafb83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.301243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] The token cache is being invalidate because project 0634338dc25f45929a4ce9871d7ac747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:06:11.301536 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.302197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.303831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1080/2166] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:06:11 2022] DELETE /identity/v3/projects/0634338dc25f45929a4ce9871d7ac747 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:06:11.307620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:11.309193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:11.322053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1087/2167] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:11.331497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67c2f9-6d52-4b4c-a927-50acc7cb7040 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:11.332935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fb67c2f9-6d52-4b4c-a927-50acc7cb7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:11.391405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed274f-2736-4cdd-b6a6-06781ebafb83 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:11.397346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:11.403643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67c2f9-6d52-4b4c-a927-50acc7cb7040 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:11.408940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67efbc-4646-47f9-86f2-b72d8ee42945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:11.409261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:11.409569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:11.409855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] PATH_INFO: `/v3/users/afaa42a0adff442cad7c15b7b452b837` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:11.410114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67efbc-4646-47f9-86f2-b72d8ee42945 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:11.410412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67efbc-4646-47f9-86f2-b72d8ee42945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:11.410794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67efbc-4646-47f9-86f2-b72d8ee42945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:11.414648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afaa42a0adff442cad7c15b7b452b837)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:11.416738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:11.426007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67efbc-4646-47f9-86f2-b72d8ee42945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:11.427642 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67efbc-4646-47f9-86f2-b72d8ee42945 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:11.448463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1088/2168] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:06:11.495884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afaa42a0adff442cad7c15b7b452b837'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.512828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afaa42a0adff442cad7c15b7b452b837'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.513573 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afaa42a0adff442cad7c15b7b452b837'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.522129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afaa42a0adff442cad7c15b7b452b837'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.522845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-74ae0c8d-2125-4977-a5b4-70fbb8a90bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afaa42a0adff442cad7c15b7b452b837'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:11.526121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:11.541968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1081/2169] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:06:11 2022] DELETE /identity/v3/users/afaa42a0adff442cad7c15b7b452b837 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:06:11.913250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67efbc-4646-47f9-86f2-b72d8ee42945 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:11.925703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67efbc-4646-47f9-86f2-b72d8ee42945 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.000456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.001576 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.001923 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.002287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] PATH_INFO: `/v3/projects/70ba8039032d4e0fa9dd06b1a6b344e7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.003684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70ba8039032d4e0fa9dd06b1a6b344e7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.005617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.066491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70ba8039032d4e0fa9dd06b1a6b344e7', 'request_id': 'req-48035a50-b1bf-4372-b1d2-f3ef8306b126'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:12.076594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70ba8039032d4e0fa9dd06b1a6b344e7', 'request_id': 'req-48035a50-b1bf-4372-b1d2-f3ef8306b126'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:12.099231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70ba8039032d4e0fa9dd06b1a6b344e7', 'request_id': 'req-48035a50-b1bf-4372-b1d2-f3ef8306b126'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:12.107017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:12.113066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] The token cache is being invalidate because project 70ba8039032d4e0fa9dd06b1a6b344e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:06:12.113785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:12.114609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-48035a50-b1bf-4372-b1d2-f3ef8306b126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:12.117124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1089/2170] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:06:11 2022] DELETE /identity/v3/projects/70ba8039032d4e0fa9dd06b1a6b344e7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:12.122342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea15e9-348d-4d6d-aa1b-ed1c8a1a0aba None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.122342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea15e9-348d-4d6d-aa1b-ed1c8a1a0aba None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.122925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea15e9-348d-4d6d-aa1b-ed1c8a1a0aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.142684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-22ea15e9-348d-4d6d-aa1b-ed1c8a1a0aba None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:12.143998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-22ea15e9-348d-4d6d-aa1b-ed1c8a1a0aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:12.213568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1082/2171] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:12.221824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7673-1c9e-4407-a598-1c3b0d43c35d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.222115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7673-1c9e-4407-a598-1c3b0d43c35d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.222395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7673-1c9e-4407-a598-1c3b0d43c35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.241728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3c1e7673-1c9e-4407-a598-1c3b0d43c35d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:12.243062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3c1e7673-1c9e-4407-a598-1c3b0d43c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:12.251869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1090/2172] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:12.256464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea15e9-348d-4d6d-aa1b-ed1c8a1a0aba None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.267980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea15e9-348d-4d6d-aa1b-ed1c8a1a0aba None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.273086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.273834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.274194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.274543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.275345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:06:12.275729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.277447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.292747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1083/2173] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 12:06:12.293896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e7673-1c9e-4407-a598-1c3b0d43c35d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.304127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e7673-1c9e-4407-a598-1c3b0d43c35d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.309124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8b7b0-c6a0-49b7-a728-f1df089dca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.309794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8b7b0-c6a0-49b7-a728-f1df089dca27 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.310088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8b7b0-c6a0-49b7-a728-f1df089dca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.310371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8b7b0-c6a0-49b7-a728-f1df089dca27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.311071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8b7b0-c6a0-49b7-a728-f1df089dca27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.313638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8b7b0-c6a0-49b7-a728-f1df089dca27 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.340812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1091/2174] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:06:12 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:06:12.344999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.357884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a144d-dc12-4b01-adfa-0aba500308ac admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.427204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df935cad-7356-4e74-a58c-8ce8b79b3c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.427904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df935cad-7356-4e74-a58c-8ce8b79b3c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.428218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df935cad-7356-4e74-a58c-8ce8b79b3c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.428503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df935cad-7356-4e74-a58c-8ce8b79b3c79 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.429315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df935cad-7356-4e74-a58c-8ce8b79b3c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.431276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df935cad-7356-4e74-a58c-8ce8b79b3c79 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.453271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a715-822c-447c-aa9c-cef832db5eea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.454320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a715-822c-447c-aa9c-cef832db5eea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.454626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a715-822c-447c-aa9c-cef832db5eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.459033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1084/2175] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:06:12 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:06:12.462975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df935cad-7356-4e74-a58c-8ce8b79b3c79 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.475386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df935cad-7356-4e74-a58c-8ce8b79b3c79 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.483066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a89aa-2580-4ecf-ab9a-2fb3f128bcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.483976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a89aa-2580-4ecf-ab9a-2fb3f128bcbf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.485106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a89aa-2580-4ecf-ab9a-2fb3f128bcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.485820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a89aa-2580-4ecf-ab9a-2fb3f128bcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.487067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a89aa-2580-4ecf-ab9a-2fb3f128bcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.489007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a89aa-2580-4ecf-ab9a-2fb3f128bcbf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.491572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7940a715-822c-447c-aa9c-cef832db5eea None None] MFA Rules not processed for user `1d4c83a1ccfc43beb3f48e4462b06794`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:12.493176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7940a715-822c-447c-aa9c-cef832db5eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:12.501150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1085/2176] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:06:12.505170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a89aa-2580-4ecf-ab9a-2fb3f128bcbf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.515558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a89aa-2580-4ecf-ab9a-2fb3f128bcbf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.522337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23de1-110a-478e-bc4a-6f10f45b0b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.523017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23de1-110a-478e-bc4a-6f10f45b0b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.523302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23de1-110a-478e-bc4a-6f10f45b0b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.523715 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c23de1-110a-478e-bc4a-6f10f45b0b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.524507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c23de1-110a-478e-bc4a-6f10f45b0b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.526153 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c23de1-110a-478e-bc4a-6f10f45b0b42 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.541878 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1086/2177] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:06:12.543454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23de1-110a-478e-bc4a-6f10f45b0b42 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.556305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c23de1-110a-478e-bc4a-6f10f45b0b42 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.561096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa520a-eca1-4cc6-8653-0bc664693c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.561993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa520a-eca1-4cc6-8653-0bc664693c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.562274 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa520a-eca1-4cc6-8653-0bc664693c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.563201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa520a-eca1-4cc6-8653-0bc664693c16 admin admin] PATH_INFO: `/v3/projects/e4fa65cbc313461c93b6099864141c5d/users/a564e5ad0dcc4a01ae71b92b1a3c1976/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.580028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa520a-eca1-4cc6-8653-0bc664693c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fa65cbc313461c93b6099864141c5d, user_id=a564e5ad0dcc4a01ae71b92b1a3c1976, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.582730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa520a-eca1-4cc6-8653-0bc664693c16 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.586484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1092/2178] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:12.595591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940a715-822c-447c-aa9c-cef832db5eea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.596824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7940a715-822c-447c-aa9c-cef832db5eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:12.601886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1087/2179] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:06:12 2022] PUT /identity/v3/projects/e4fa65cbc313461c93b6099864141c5d/users/a564e5ad0dcc4a01ae71b92b1a3c1976/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:06:12.612543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff736fcb-74f8-4600-9bdc-2b96c370c288 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.613081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff736fcb-74f8-4600-9bdc-2b96c370c288 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.613468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff736fcb-74f8-4600-9bdc-2b96c370c288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.648660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ff736fcb-74f8-4600-9bdc-2b96c370c288 None None] MFA Rules not processed for user `a564e5ad0dcc4a01ae71b92b1a3c1976`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:12.649992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ff736fcb-74f8-4600-9bdc-2b96c370c288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:12.663116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:12.663395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940a715-822c-447c-aa9c-cef832db5eea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.720637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce88eb3-a798-4862-8d86-00e104ac68b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.721306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce88eb3-a798-4862-8d86-00e104ac68b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.721599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce88eb3-a798-4862-8d86-00e104ac68b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.721884 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce88eb3-a798-4862-8d86-00e104ac68b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.724133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1088/2180] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:06:12.728637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff736fcb-74f8-4600-9bdc-2b96c370c288 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.763097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:12.802328 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce88eb3-a798-4862-8d86-00e104ac68b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.804012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce88eb3-a798-4862-8d86-00e104ac68b9 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.814148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff736fcb-74f8-4600-9bdc-2b96c370c288 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.815572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1093/2181] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:06:12.830070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3f4a-fc9e-4ba9-9ca2-73dec00bb075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.831178 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3f4a-fc9e-4ba9-9ca2-73dec00bb075 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.831562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3f4a-fc9e-4ba9-9ca2-73dec00bb075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.832076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf3f4a-fc9e-4ba9-9ca2-73dec00bb075 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.832926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3f4a-fc9e-4ba9-9ca2-73dec00bb075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.835125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf3f4a-fc9e-4ba9-9ca2-73dec00bb075 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.872216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1089/2182] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:06:12 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:06:12.873829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce88eb3-a798-4862-8d86-00e104ac68b9 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:12.890084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce88eb3-a798-4862-8d86-00e104ac68b9 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:12.964120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ee9f3-1b84-4216-bb15-113218819942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:12.965153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee9f3-1b84-4216-bb15-113218819942 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:12.965536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee9f3-1b84-4216-bb15-113218819942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:12.966110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee9f3-1b84-4216-bb15-113218819942 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:12.967033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ee9f3-1b84-4216-bb15-113218819942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:12.969101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ee9f3-1b84-4216-bb15-113218819942 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:12.994464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1094/2183] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:06:12 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:06:12.998443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3f4a-fc9e-4ba9-9ca2-73dec00bb075 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:13.010653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf3f4a-fc9e-4ba9-9ca2-73dec00bb075 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:13.016508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b8a2-e642-4324-a263-7619468b0174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:13.017249 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b8a2-e642-4324-a263-7619468b0174 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:13.017540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b8a2-e642-4324-a263-7619468b0174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:13.017824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b8a2-e642-4324-a263-7619468b0174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:13.019045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5b8a2-e642-4324-a263-7619468b0174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:13.020584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5b8a2-e642-4324-a263-7619468b0174 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:13.036219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1090/2184] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:06:13.038594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ee9f3-1b84-4216-bb15-113218819942 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:13.052521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ee9f3-1b84-4216-bb15-113218819942 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:13.060684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e538d-e1dd-4444-88cd-431418c34397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:13.062236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e538d-e1dd-4444-88cd-431418c34397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:13.062656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e538d-e1dd-4444-88cd-431418c34397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:13.062945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e538d-e1dd-4444-88cd-431418c34397 admin admin] PATH_INFO: `/v3/projects/c2f2e0a31dbf4d8fbb929451fee211e8/users/f2b8310b6e6b478fa99d37ea7feff31f/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:13.083833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e538d-e1dd-4444-88cd-431418c34397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f2e0a31dbf4d8fbb929451fee211e8, user_id=f2b8310b6e6b478fa99d37ea7feff31f, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:13.086955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e538d-e1dd-4444-88cd-431418c34397 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:13.106710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1095/2185] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:06:13 2022] PUT /identity/v3/projects/c2f2e0a31dbf4d8fbb929451fee211e8/users/f2b8310b6e6b478fa99d37ea7feff31f/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:13.114157 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b8a2-e642-4324-a263-7619468b0174 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:13.136128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b8a2-e642-4324-a263-7619468b0174 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:13.231144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04aa26-642d-4895-a704-bb6e039f502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:13.231862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04aa26-642d-4895-a704-bb6e039f502a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:13.232388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04aa26-642d-4895-a704-bb6e039f502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:13.232881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04aa26-642d-4895-a704-bb6e039f502a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:13.233679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04aa26-642d-4895-a704-bb6e039f502a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:13.236456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04aa26-642d-4895-a704-bb6e039f502a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:13.260447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1091/2186] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:06:13.264611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e538d-e1dd-4444-88cd-431418c34397 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:13.275709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e538d-e1dd-4444-88cd-431418c34397 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:13.285244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9708-390f-4531-9eb4-8b2209c16ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:13.286170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9708-390f-4531-9eb4-8b2209c16ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:13.286458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9708-390f-4531-9eb4-8b2209c16ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:13.286741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f9708-390f-4531-9eb4-8b2209c16ac9 admin admin] PATH_INFO: `/v3/projects/c2f2e0a31dbf4d8fbb929451fee211e8/users/f2b8310b6e6b478fa99d37ea7feff31f/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:13.289036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f9708-390f-4531-9eb4-8b2209c16ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f2e0a31dbf4d8fbb929451fee211e8, user_id=f2b8310b6e6b478fa99d37ea7feff31f, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:13.291929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f9708-390f-4531-9eb4-8b2209c16ac9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:13.306898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1096/2187] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:06:13 2022] PUT /identity/v3/projects/c2f2e0a31dbf4d8fbb929451fee211e8/users/f2b8310b6e6b478fa99d37ea7feff31f/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:13.319214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464919e3-55f5-438a-8167-1c2e68d67049 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:13.319885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464919e3-55f5-438a-8167-1c2e68d67049 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:13.320558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464919e3-55f5-438a-8167-1c2e68d67049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:13.358814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-464919e3-55f5-438a-8167-1c2e68d67049 None None] MFA Rules not processed for user `f2b8310b6e6b478fa99d37ea7feff31f`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:13.361464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-464919e3-55f5-438a-8167-1c2e68d67049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:13.375602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:13.442663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1092/2188] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:13.453633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9708-390f-4531-9eb4-8b2209c16ac9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:13.512184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f9708-390f-4531-9eb4-8b2209c16ac9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:13.516189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d801-d124-408a-89b0-e81a651d2439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:13.516975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d801-d124-408a-89b0-e81a651d2439 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:13.517258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d801-d124-408a-89b0-e81a651d2439 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:13.517536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d801-d124-408a-89b0-e81a651d2439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:13.527118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5d801-d124-408a-89b0-e81a651d2439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:13.528595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5d801-d124-408a-89b0-e81a651d2439 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:13.538735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1097/2189] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:06:13.905227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464919e3-55f5-438a-8167-1c2e68d67049 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:13.969036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464919e3-55f5-438a-8167-1c2e68d67049 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:13.973670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:13.974541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:13.974827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:13.975106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] PATH_INFO: `/v3/users/a564e5ad0dcc4a01ae71b92b1a3c1976` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:13.976337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a564e5ad0dcc4a01ae71b92b1a3c1976)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:13.978282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:14.024916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a564e5ad0dcc4a01ae71b92b1a3c1976'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.036746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a564e5ad0dcc4a01ae71b92b1a3c1976'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.037421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a564e5ad0dcc4a01ae71b92b1a3c1976'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.044279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a564e5ad0dcc4a01ae71b92b1a3c1976'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.044928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a564e5ad0dcc4a01ae71b92b1a3c1976'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.058786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1093/2190] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:06:13 2022] DELETE /identity/v3/users/a564e5ad0dcc4a01ae71b92b1a3c1976 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:14.069246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4ac8e-3062-4eec-a4d0-c5712c0368b7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:14.069537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4ac8e-3062-4eec-a4d0-c5712c0368b7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:14.069824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4ac8e-3062-4eec-a4d0-c5712c0368b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:14.088987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-69b4ac8e-3062-4eec-a4d0-c5712c0368b7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:14.090197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69b4ac8e-3062-4eec-a4d0-c5712c0368b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:14.112499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:14.153466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1098/2191] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:14.163795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:14.165535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:14.198229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:14.236348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d1d83-dea5-43f0-80d1-ebb5706953a0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:14.241309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddddeeb-1a96-4811-8708-4557374b7aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:14.241975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddddeeb-1a96-4811-8708-4557374b7aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:14.242266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddddeeb-1a96-4811-8708-4557374b7aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:14.242554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddddeeb-1a96-4811-8708-4557374b7aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:14.252563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddddeeb-1a96-4811-8708-4557374b7aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:14.254322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddddeeb-1a96-4811-8708-4557374b7aa6 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:14.269143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1094/2192] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:06:14.558198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4ac8e-3062-4eec-a4d0-c5712c0368b7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:14.560020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69b4ac8e-3062-4eec-a4d0-c5712c0368b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:14.613218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4ac8e-3062-4eec-a4d0-c5712c0368b7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:14.620362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:14.621289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:14.622376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:14.622376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] PATH_INFO: `/v3/projects/e4fa65cbc313461c93b6099864141c5d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:14.623205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fa65cbc313461c93b6099864141c5d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:14.625527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:14.692429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fa65cbc313461c93b6099864141c5d', 'request_id': 'req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.702536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fa65cbc313461c93b6099864141c5d', 'request_id': 'req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.710966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fa65cbc313461c93b6099864141c5d', 'request_id': 'req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.727164 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] The token cache is being invalidate because project e4fa65cbc313461c93b6099864141c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:06:14.727542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.728325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa8488f6-3034-468a-9d4b-f3bcaa4d3f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.730771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1099/2193] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:06:14 2022] DELETE /identity/v3/projects/e4fa65cbc313461c93b6099864141c5d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:14.735140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddddeeb-1a96-4811-8708-4557374b7aa6 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:14.754936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddddeeb-1a96-4811-8708-4557374b7aa6 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:14.837726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:14.838421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:14.838734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:14.839197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] PATH_INFO: `/v3/users/f2b8310b6e6b478fa99d37ea7feff31f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:14.840441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b8310b6e6b478fa99d37ea7feff31f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:14.842497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:14.901149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b8310b6e6b478fa99d37ea7feff31f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.916932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b8310b6e6b478fa99d37ea7feff31f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.917554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b8310b6e6b478fa99d37ea7feff31f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.925446 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b8310b6e6b478fa99d37ea7feff31f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.926411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b8310b6e6b478fa99d37ea7feff31f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:14.943771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1095/2194] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:06:14 2022] DELETE /identity/v3/users/f2b8310b6e6b478fa99d37ea7feff31f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:06:15.342259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09611-942f-4442-b8fb-970252fb7f0c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:15.342570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09611-942f-4442-b8fb-970252fb7f0c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:15.342878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e09611-942f-4442-b8fb-970252fb7f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:15.367964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-21e09611-942f-4442-b8fb-970252fb7f0c None None] MFA Rules not processed for user `1d4c83a1ccfc43beb3f48e4462b06794`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:15.369686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-21e09611-942f-4442-b8fb-970252fb7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:15.435479 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:15.438565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:15.445796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1100/2195] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:15.529939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:15.557326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bffd0d-a6c6-4864-b214-542ca3ba7661 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:15.562596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:15.563545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:15.563909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:15.564230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] PATH_INFO: `/v3/projects/c2f2e0a31dbf4d8fbb929451fee211e8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:15.565521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f2e0a31dbf4d8fbb929451fee211e8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:15.567510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:15.641150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f2e0a31dbf4d8fbb929451fee211e8', 'request_id': 'req-f4e1e203-7599-4bae-84f8-0e842b5d241a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:15.664329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f2e0a31dbf4d8fbb929451fee211e8', 'request_id': 'req-f4e1e203-7599-4bae-84f8-0e842b5d241a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:15.677933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f2e0a31dbf4d8fbb929451fee211e8', 'request_id': 'req-f4e1e203-7599-4bae-84f8-0e842b5d241a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:15.696187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] The token cache is being invalidate because project c2f2e0a31dbf4d8fbb929451fee211e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:06:15.696616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:15.697246 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:15.699316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1096/2196] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:06:15 2022] DELETE /identity/v3/projects/c2f2e0a31dbf4d8fbb929451fee211e8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:15.878371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351ece-fdaf-45d9-b180-f79d3703d603 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:15.878984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351ece-fdaf-45d9-b180-f79d3703d603 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:15.880392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351ece-fdaf-45d9-b180-f79d3703d603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:15.915487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-46351ece-fdaf-45d9-b180-f79d3703d603 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:15.916827 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-46351ece-fdaf-45d9-b180-f79d3703d603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:15.964560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:15.981732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1101/2197] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:16.030381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:16.032602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:16.131968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1e203-7599-4bae-84f8-0e842b5d241a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:16.138566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:16.139305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:16.139659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:16.140007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:16.150933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:16.152612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:16.162975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1097/2198] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:06:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:06:24.208435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd260254-b636-4690-9f3a-f371ec68182d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:24.209065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd260254-b636-4690-9f3a-f371ec68182d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:24.209065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd260254-b636-4690-9f3a-f371ec68182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:24.255020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bd260254-b636-4690-9f3a-f371ec68182d None None] MFA Rules not processed for user `166b42a5d6c04659afb20c088dfead76`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:24.260271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bd260254-b636-4690-9f3a-f371ec68182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:24.376285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1102/2199] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:25.968836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:25.971088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:26.065403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0dddd-fa76-4820-8845-7037cd61c0c7 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:26.069711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:26.070674 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:26.071032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:26.071321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] PATH_INFO: `/v3/users/317b7259e0344f238a06da0521ee348a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:26.072639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=317b7259e0344f238a06da0521ee348a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:26.074530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:26.092055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:26.150876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '317b7259e0344f238a06da0521ee348a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:26.170568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '317b7259e0344f238a06da0521ee348a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:26.172421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '317b7259e0344f238a06da0521ee348a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:26.183701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '317b7259e0344f238a06da0521ee348a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:26.185009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '317b7259e0344f238a06da0521ee348a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:26.202024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1098/2200] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:06:25 2022] DELETE /identity/v3/users/317b7259e0344f238a06da0521ee348a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:26.206551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e168-55d5-453b-864b-56fb869f0ee8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:26.207081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e168-55d5-453b-864b-56fb869f0ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:26.207177 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e168-55d5-453b-864b-56fb869f0ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:26.259896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b245e168-55d5-453b-864b-56fb869f0ee8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:26.261404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b245e168-55d5-453b-864b-56fb869f0ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:26.335336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1103/2201] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:26.345325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:26.347135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:26.432787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a140e14-8a16-49b8-9b3c-caf64dc57516 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:26.449774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07389be3-553f-4988-be1d-d47c6701e214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:26.450641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07389be3-553f-4988-be1d-d47c6701e214 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:26.451033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07389be3-553f-4988-be1d-d47c6701e214 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:26.451421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07389be3-553f-4988-be1d-d47c6701e214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:26.469397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07389be3-553f-4988-be1d-d47c6701e214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:26.473708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07389be3-553f-4988-be1d-d47c6701e214 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:26.510469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1099/2202] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:06:27.354583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245e168-55d5-453b-864b-56fb869f0ee8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:27.357049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b245e168-55d5-453b-864b-56fb869f0ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:27.391952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245e168-55d5-453b-864b-56fb869f0ee8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:27.413479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:27.414579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:27.415355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:27.415921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] PATH_INFO: `/v3/projects/701bba02c995487f81e17bd916730396` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:27.417542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=701bba02c995487f81e17bd916730396)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:27.419738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:27.505431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701bba02c995487f81e17bd916730396', 'request_id': 'req-131ade4d-35ea-481e-a311-b0c9acef6b8e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:27.518572 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701bba02c995487f81e17bd916730396', 'request_id': 'req-131ade4d-35ea-481e-a311-b0c9acef6b8e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:27.528935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701bba02c995487f81e17bd916730396', 'request_id': 'req-131ade4d-35ea-481e-a311-b0c9acef6b8e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:27.543397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] The token cache is being invalidate because project 701bba02c995487f81e17bd916730396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:06:27.544350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:27.545516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-131ade4d-35ea-481e-a311-b0c9acef6b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:27.548892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1104/2203] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:06:27 2022] DELETE /identity/v3/projects/701bba02c995487f81e17bd916730396 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:30.524451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454ee36-e44a-4812-a386-7f2c11a7d19d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:30.525013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454ee36-e44a-4812-a386-7f2c11a7d19d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:30.525013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454ee36-e44a-4812-a386-7f2c11a7d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:30.543809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3454ee36-e44a-4812-a386-7f2c11a7d19d None None] MFA Rules not processed for user `7575179edbda45208561c983fc5b956c`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:30.545111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3454ee36-e44a-4812-a386-7f2c11a7d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:30.608418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1100/2204] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:37.539308 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:38.168376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ada865-d3be-4719-8438-5beea9ca2693 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:38.168668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ada865-d3be-4719-8438-5beea9ca2693 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:38.168955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ada865-d3be-4719-8438-5beea9ca2693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:38.190510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a1ada865-d3be-4719-8438-5beea9ca2693 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:38.191660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a1ada865-d3be-4719-8438-5beea9ca2693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:38.265098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1105/2205] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:06:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:38.279970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454ee36-e44a-4812-a386-7f2c11a7d19d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:38.282887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3454ee36-e44a-4812-a386-7f2c11a7d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:38.354581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:38.384992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454ee36-e44a-4812-a386-7f2c11a7d19d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:38.390534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb142437-1ab5-40a1-8d60-38547ea4acb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:38.391185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb142437-1ab5-40a1-8d60-38547ea4acb3 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:38.391471 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb142437-1ab5-40a1-8d60-38547ea4acb3 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:38.391748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb142437-1ab5-40a1-8d60-38547ea4acb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:38.400244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb142437-1ab5-40a1-8d60-38547ea4acb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:38.401841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb142437-1ab5-40a1-8d60-38547ea4acb3 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:38.409956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1101/2206] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:06:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:06:45.993079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bddf0-1e51-40fa-a53e-982594acd47c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:45.994190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bddf0-1e51-40fa-a53e-982594acd47c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:45.994852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bddf0-1e51-40fa-a53e-982594acd47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:46.039664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f92bddf0-1e51-40fa-a53e-982594acd47c None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:46.040900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f92bddf0-1e51-40fa-a53e-982594acd47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:46.068861 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1106/2207] 10.222.0.32 () {58 vars in 967 bytes} [Mon May 30 12:06:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:46.079936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb142437-1ab5-40a1-8d60-38547ea4acb3 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:46.091462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb142437-1ab5-40a1-8d60-38547ea4acb3 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:46.098632 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979095c2-faa9-482d-ba95-7f68dc7d72a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:46.099692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979095c2-faa9-482d-ba95-7f68dc7d72a0 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:46.100242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979095c2-faa9-482d-ba95-7f68dc7d72a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:46.100823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979095c2-faa9-482d-ba95-7f68dc7d72a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:46.133937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979095c2-faa9-482d-ba95-7f68dc7d72a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:46.135770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979095c2-faa9-482d-ba95-7f68dc7d72a0 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:46.151943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1102/2208] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:06:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:06:48.462066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bddf0-1e51-40fa-a53e-982594acd47c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:48.463653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f92bddf0-1e51-40fa-a53e-982594acd47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:48.541264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bddf0-1e51-40fa-a53e-982594acd47c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:48.546723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:48.548105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:48.548496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:48.549356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] PATH_INFO: `/v3/users/a440b40bc3124bafa502f66a004d0de0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:48.552412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a440b40bc3124bafa502f66a004d0de0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:48.556606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:48.615425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a440b40bc3124bafa502f66a004d0de0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:48.629072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a440b40bc3124bafa502f66a004d0de0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:48.629810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a440b40bc3124bafa502f66a004d0de0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:48.630373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a440b40bc3124bafa502f66a004d0de0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:48.637773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-75afb605-df50-442d-a2dd-2e14594f0d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a440b40bc3124bafa502f66a004d0de0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:48.652753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1107/2209] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:06:48 2022] DELETE /identity/v3/users/a440b40bc3124bafa502f66a004d0de0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:48.945056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979095c2-faa9-482d-ba95-7f68dc7d72a0 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:48.957895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979095c2-faa9-482d-ba95-7f68dc7d72a0 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:49.049857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:49.051368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.051902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.052429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] PATH_INFO: `/v3/projects/b7843d3fb7cb402b8a85179da7bfffaa` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.054128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7843d3fb7cb402b8a85179da7bfffaa)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:49.056976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:49.113861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7843d3fb7cb402b8a85179da7bfffaa', 'request_id': 'req-97f7a04f-9c77-48ee-8fe9-8869b71cc968'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:49.120890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7843d3fb7cb402b8a85179da7bfffaa', 'request_id': 'req-97f7a04f-9c77-48ee-8fe9-8869b71cc968'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:49.127498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7843d3fb7cb402b8a85179da7bfffaa', 'request_id': 'req-97f7a04f-9c77-48ee-8fe9-8869b71cc968'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:49.139900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] The token cache is being invalidate because project b7843d3fb7cb402b8a85179da7bfffaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:06:49.140227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:49.140880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-97f7a04f-9c77-48ee-8fe9-8869b71cc968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:06:49.142452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1103/2210] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:06:48 2022] DELETE /identity/v3/projects/b7843d3fb7cb402b8a85179da7bfffaa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:49.147895 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5391de-a10e-436a-bf70-efd54c274b82 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.148245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5391de-a10e-436a-bf70-efd54c274b82 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.148565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5391de-a10e-436a-bf70-efd54c274b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.167384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-af5391de-a10e-436a-bf70-efd54c274b82 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:49.168658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af5391de-a10e-436a-bf70-efd54c274b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:49.176183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:49.235562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1108/2211] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:49.244139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8409fc0-a91b-4e55-a0c7-315f7d0ea64b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.244541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8409fc0-a91b-4e55-a0c7-315f7d0ea64b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.244897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8409fc0-a91b-4e55-a0c7-315f7d0ea64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.264852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e8409fc0-a91b-4e55-a0c7-315f7d0ea64b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:49.266199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e8409fc0-a91b-4e55-a0c7-315f7d0ea64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:49.274825 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1104/2212] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:49.279122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5391de-a10e-436a-bf70-efd54c274b82 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:49.288844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5391de-a10e-436a-bf70-efd54c274b82 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:49.293837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:49.294523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.294815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.295100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.296411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:06:49.296732 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:49.298419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:49.310717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1109/2213] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:06:49.315224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8409fc0-a91b-4e55-a0c7-315f7d0ea64b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:49.324969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8409fc0-a91b-4e55-a0c7-315f7d0ea64b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:49.330219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa5c83-a8f1-4bdc-ba6e-335319f0e42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:49.330894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa5c83-a8f1-4bdc-ba6e-335319f0e42f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.331599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa5c83-a8f1-4bdc-ba6e-335319f0e42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.331599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa5c83-a8f1-4bdc-ba6e-335319f0e42f admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.332182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa5c83-a8f1-4bdc-ba6e-335319f0e42f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:49.334172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa5c83-a8f1-4bdc-ba6e-335319f0e42f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:49.366375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1105/2214] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:06:49 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:06:49.367852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:49.379420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b0abc-b4cd-4aa9-8ece-884793450728 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:49.443646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b331d7-bedc-4828-8a09-54378b9a6ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:49.443646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b331d7-bedc-4828-8a09-54378b9a6ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.443646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b331d7-bedc-4828-8a09-54378b9a6ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.443646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b331d7-bedc-4828-8a09-54378b9a6ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.444125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b331d7-bedc-4828-8a09-54378b9a6ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:49.445073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b331d7-bedc-4828-8a09-54378b9a6ef2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:49.471504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1110/2215] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:06:49 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:06:49.472784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa5c83-a8f1-4bdc-ba6e-335319f0e42f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:49.484048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa5c83-a8f1-4bdc-ba6e-335319f0e42f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:49.488856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871605c-014c-4e8d-9d97-8d96f99db42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:49.489527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871605c-014c-4e8d-9d97-8d96f99db42e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.489814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871605c-014c-4e8d-9d97-8d96f99db42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.490291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871605c-014c-4e8d-9d97-8d96f99db42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.491033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871605c-014c-4e8d-9d97-8d96f99db42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:49.492687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871605c-014c-4e8d-9d97-8d96f99db42e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:49.507940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1106/2216] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:06:49.509279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b331d7-bedc-4828-8a09-54378b9a6ef2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:49.521795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b331d7-bedc-4828-8a09-54378b9a6ef2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:49.526547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4966-f684-406d-904f-14189aa9052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:49.527226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4966-f684-406d-904f-14189aa9052f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.527517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4966-f684-406d-904f-14189aa9052f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.527807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4966-f684-406d-904f-14189aa9052f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.528573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb4966-f684-406d-904f-14189aa9052f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:49.530714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb4966-f684-406d-904f-14189aa9052f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:49.541908 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1111/2217] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:06:49.548711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871605c-014c-4e8d-9d97-8d96f99db42e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:49.556423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:49.561618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871605c-014c-4e8d-9d97-8d96f99db42e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:49.568506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:49.569421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.569711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.569995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] PATH_INFO: `/v3/projects/f0fffd99d1604110aa9896117647ca3e/users/27f41db070cf453985a59132aad1bb03/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.598501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0fffd99d1604110aa9896117647ca3e, user_id=27f41db070cf453985a59132aad1bb03, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:49.602707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:49.628150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1107/2218] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:06:49 2022] PUT /identity/v3/projects/f0fffd99d1604110aa9896117647ca3e/users/27f41db070cf453985a59132aad1bb03/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:06:49.634326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af452385-a337-429e-882c-0008ae70bc32 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.635551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af452385-a337-429e-882c-0008ae70bc32 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.635920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af452385-a337-429e-882c-0008ae70bc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.673594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-af452385-a337-429e-882c-0008ae70bc32 None None] MFA Rules not processed for user `27f41db070cf453985a59132aad1bb03`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:49.674886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af452385-a337-429e-882c-0008ae70bc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:49.761604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1112/2219] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:06:49.770456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:49.772179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:49.853687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ed3fb-68d0-49a1-8ec9-b92f6db86ce2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:49.857434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f973fae-bdaf-4b79-98d8-7eb180b3c3ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:49.858303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f973fae-bdaf-4b79-98d8-7eb180b3c3ed service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:49.858590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f973fae-bdaf-4b79-98d8-7eb180b3c3ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:49.858866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f973fae-bdaf-4b79-98d8-7eb180b3c3ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:49.867798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f973fae-bdaf-4b79-98d8-7eb180b3c3ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:49.869285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f973fae-bdaf-4b79-98d8-7eb180b3c3ed service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:49.878550 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1108/2220] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:06:53.874877 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9931a-b47c-4471-b2f8-47af7c5adb03 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:53.875419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9931a-b47c-4471-b2f8-47af7c5adb03 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:53.875419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9931a-b47c-4471-b2f8-47af7c5adb03 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:53.876916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1113/2221] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 12:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:06:53.884739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d2a70-bf3f-4096-a8ed-ab6c30e40915 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:53.885256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d2a70-bf3f-4096-a8ed-ab6c30e40915 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:53.885405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d2a70-bf3f-4096-a8ed-ab6c30e40915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:53.937370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-281d2a70-bf3f-4096-a8ed-ab6c30e40915 None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:53.938618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-281d2a70-bf3f-4096-a8ed-ab6c30e40915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:54.017527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1109/2222] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 12:06:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:54.031388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9931a-b47c-4471-b2f8-47af7c5adb03 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:54.033094 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-99d9931a-b47c-4471-b2f8-47af7c5adb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:54.043534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:06:54.102136 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9931a-b47c-4471-b2f8-47af7c5adb03 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:54.108847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3693f0-1b18-4238-95ba-dc330a7c9a2f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad37bdce39f64836b0cd14613cabe91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:54.109890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3693f0-1b18-4238-95ba-dc330a7c9a2f service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:54.110182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3693f0-1b18-4238-95ba-dc330a7c9a2f service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:54.110499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3693f0-1b18-4238-95ba-dc330a7c9a2f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:54.131739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3693f0-1b18-4238-95ba-dc330a7c9a2f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:54.133637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3693f0-1b18-4238-95ba-dc330a7c9a2f service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:54.146358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1114/2223] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:06:54.235706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8956d5d-5982-49ab-8f15-03aeae6104b7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:54.236669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8956d5d-5982-49ab-8f15-03aeae6104b7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:54.239095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8956d5d-5982-49ab-8f15-03aeae6104b7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:54.241343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1110/2224] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 12:06:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:06:54.246427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674c497-a888-43c1-8f5a-52a6238ed790 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:54.247281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674c497-a888-43c1-8f5a-52a6238ed790 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:54.247281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674c497-a888-43c1-8f5a-52a6238ed790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:54.315354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b674c497-a888-43c1-8f5a-52a6238ed790 None None] MFA Rules not processed for user `15ea38389eee4472a8917023643be9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:54.316598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b674c497-a888-43c1-8f5a-52a6238ed790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:54.372348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1115/2225] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:06:54.384659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8956d5d-5982-49ab-8f15-03aeae6104b7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:54.386375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f8956d5d-5982-49ab-8f15-03aeae6104b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:54.401084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8956d5d-5982-49ab-8f15-03aeae6104b7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:54.404761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e1406-87f4-4181-b283-01abfbe9b5cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:54.405505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e1406-87f4-4181-b283-01abfbe9b5cd service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:54.405863 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e1406-87f4-4181-b283-01abfbe9b5cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:54.406215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e1406-87f4-4181-b283-01abfbe9b5cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:54.414254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e1406-87f4-4181-b283-01abfbe9b5cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:54.415820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e1406-87f4-4181-b283-01abfbe9b5cd service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:54.424454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1111/2226] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:06:54.509881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ef8cc-97cc-4f41-a1f3-3a9666843514 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:54.510182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ef8cc-97cc-4f41-a1f3-3a9666843514 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:54.510472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ef8cc-97cc-4f41-a1f3-3a9666843514 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:54.512139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1116/2227] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 12:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:06:54.518320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ace24-fe6f-45d3-89c5-64c6a237e48d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:54.518672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ace24-fe6f-45d3-89c5-64c6a237e48d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:54.519126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ace24-fe6f-45d3-89c5-64c6a237e48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:54.540117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-266ace24-fe6f-45d3-89c5-64c6a237e48d None None] MFA Rules not processed for user `70dce2f0284847ab8f07f55d700d0f60`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:06:54.541371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-266ace24-fe6f-45d3-89c5-64c6a237e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:06:54.550797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1112/2228] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:06:54.561770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ef8cc-97cc-4f41-a1f3-3a9666843514 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:06:54.573050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ef8cc-97cc-4f41-a1f3-3a9666843514 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:06:54.578655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed4662-af39-4935-82a5-8355731cd88e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:06:54.579434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed4662-af39-4935-82a5-8355731cd88e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:06:54.579722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed4662-af39-4935-82a5-8355731cd88e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:06:54.580016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed4662-af39-4935-82a5-8355731cd88e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:06:54.605227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed4662-af39-4935-82a5-8355731cd88e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:06:54.607034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed4662-af39-4935-82a5-8355731cd88e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:06:54.619783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1117/2229] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:06:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:07:00.152319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ace24-fe6f-45d3-89c5-64c6a237e48d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:07:00.157349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:07:00.221240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ace24-fe6f-45d3-89c5-64c6a237e48d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:07:00.227707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:07:00.228746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:00.229076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:00.229491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] PATH_INFO: `/v3/users/27f41db070cf453985a59132aad1bb03` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:00.230765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f41db070cf453985a59132aad1bb03)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:07:00.232814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:07:00.283923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f41db070cf453985a59132aad1bb03'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.296328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f41db070cf453985a59132aad1bb03'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.297023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f41db070cf453985a59132aad1bb03'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.303995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f41db070cf453985a59132aad1bb03'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.304675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f41db070cf453985a59132aad1bb03'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.320459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1113/2230] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:07:00 2022] DELETE /identity/v3/users/27f41db070cf453985a59132aad1bb03 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:07:00.326062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfaf973-ac04-41ab-a701-3dc4aa2b9be0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:00.327205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfaf973-ac04-41ab-a701-3dc4aa2b9be0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:00.327396 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfaf973-ac04-41ab-a701-3dc4aa2b9be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:00.347418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8dfaf973-ac04-41ab-a701-3dc4aa2b9be0 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:07:00.348783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8dfaf973-ac04-41ab-a701-3dc4aa2b9be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:00.413773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1118/2231] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:07:00.423644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:07:00.425071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:00.488304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b5046-6ee9-4a86-91f1-ebf51d969a1d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:07:00.492106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc524051-19bf-4b0d-9ea8-07747f87f629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:07:00.492667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc524051-19bf-4b0d-9ea8-07747f87f629 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:00.492954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc524051-19bf-4b0d-9ea8-07747f87f629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:00.493233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc524051-19bf-4b0d-9ea8-07747f87f629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:00.503148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc524051-19bf-4b0d-9ea8-07747f87f629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:07:00.504638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc524051-19bf-4b0d-9ea8-07747f87f629 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:07:00.515528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1114/2232] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:07:00.825050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfaf973-ac04-41ab-a701-3dc4aa2b9be0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:07:00.827018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8dfaf973-ac04-41ab-a701-3dc4aa2b9be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:00.843727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfaf973-ac04-41ab-a701-3dc4aa2b9be0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:07:00.852488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:07:00.854191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:00.854449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:00.854707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] PATH_INFO: `/v3/projects/f0fffd99d1604110aa9896117647ca3e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:00.856787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0fffd99d1604110aa9896117647ca3e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:07:00.860528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:07:00.906277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:07:00.922882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0fffd99d1604110aa9896117647ca3e', 'request_id': 'req-14fa2b6c-f82e-4edf-b788-a55a7118638e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.929963 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0fffd99d1604110aa9896117647ca3e', 'request_id': 'req-14fa2b6c-f82e-4edf-b788-a55a7118638e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.937400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0fffd99d1604110aa9896117647ca3e', 'request_id': 'req-14fa2b6c-f82e-4edf-b788-a55a7118638e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.949867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] The token cache is being invalidate because project f0fffd99d1604110aa9896117647ca3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:07:00.950184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.950866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:07:00.952787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1119/2233] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:07:00 2022] DELETE /identity/v3/projects/f0fffd99d1604110aa9896117647ca3e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:07:04.925843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a8535-171d-482f-96a5-13cc3a1c57ab None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:04.926362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a8535-171d-482f-96a5-13cc3a1c57ab None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:04.926474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a8535-171d-482f-96a5-13cc3a1c57ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:04.962618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:07:04.981426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5d8a8535-171d-482f-96a5-13cc3a1c57ab None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:07:04.983249 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5d8a8535-171d-482f-96a5-13cc3a1c57ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:05.057548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1115/2234] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:07:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:07:05.074302 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:07:05.075807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:05.185124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa2b6c-f82e-4edf-b788-a55a7118638e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:07:05.198666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c20db-03aa-465e-91ff-9a51dcdb87ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:07:05.199348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c20db-03aa-465e-91ff-9a51dcdb87ee service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:05.199690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c20db-03aa-465e-91ff-9a51dcdb87ee service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:05.199906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c20db-03aa-465e-91ff-9a51dcdb87ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:05.212982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c20db-03aa-465e-91ff-9a51dcdb87ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:07:05.212982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c20db-03aa-465e-91ff-9a51dcdb87ee service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:07:05.224012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1120/2235] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:07:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 30 12:07:15.053662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=29 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:07:15.219713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=55,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:07:18.647590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a8535-171d-482f-96a5-13cc3a1c57ab None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:07:18.649052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5d8a8535-171d-482f-96a5-13cc3a1c57ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:18.758608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a8535-171d-482f-96a5-13cc3a1c57ab None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:07:18.763829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3629683-e592-4434-9a99-3121b6db2d10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:07:18.764516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3629683-e592-4434-9a99-3121b6db2d10 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:18.764819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3629683-e592-4434-9a99-3121b6db2d10 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:18.765087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3629683-e592-4434-9a99-3121b6db2d10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:18.767067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a3629683-e592-4434-9a99-3121b6db2d10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:18.783747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3629683-e592-4434-9a99-3121b6db2d10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:07:18.785463 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3629683-e592-4434-9a99-3121b6db2d10 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:07:18.800475 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1116/2236] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:07:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:07:28.792884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=41 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:07:34.003269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c704e89-6ab7-4983-8e4e-49ad7027c481 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:34.004104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c704e89-6ab7-4983-8e4e-49ad7027c481 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:34.004104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c704e89-6ab7-4983-8e4e-49ad7027c481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:34.021268 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5c704e89-6ab7-4983-8e4e-49ad7027c481 None None] MFA Rules not processed for user `1d4c83a1ccfc43beb3f48e4462b06794`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:07:34.022486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5c704e89-6ab7-4983-8e4e-49ad7027c481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:34.082095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1121/2237] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:07:35.782923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881632e-18fb-4764-bd60-8f9601250c13 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:35.783568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881632e-18fb-4764-bd60-8f9601250c13 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:35.783568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881632e-18fb-4764-bd60-8f9601250c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:35.806430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9881632e-18fb-4764-bd60-8f9601250c13 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:07:35.807665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9881632e-18fb-4764-bd60-8f9601250c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:07:35.822345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1117/2238] 10.222.0.32 () {58 vars in 967 bytes} [Mon May 30 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:07:35.845897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c704e89-6ab7-4983-8e4e-49ad7027c481 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:07:35.858562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c704e89-6ab7-4983-8e4e-49ad7027c481 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:07:35.863860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe805cf9-3506-4b1e-bb12-ce91f06246ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:07:35.864556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe805cf9-3506-4b1e-bb12-ce91f06246ab service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:07:35.864847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe805cf9-3506-4b1e-bb12-ce91f06246ab service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:07:35.865129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe805cf9-3506-4b1e-bb12-ce91f06246ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:07:35.873785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe805cf9-3506-4b1e-bb12-ce91f06246ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:07:35.875341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe805cf9-3506-4b1e-bb12-ce91f06246ab service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:07:35.885058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1122/2239] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:07:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:07:45.799316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:07:45.880840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=31 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:08:11.050377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1738-07b8-4c79-ad4c-3ac7bace8da5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:11.050945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1738-07b8-4c79-ad4c-3ac7bace8da5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:11.050945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf1738-07b8-4c79-ad4c-3ac7bace8da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:11.071583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-34bf1738-07b8-4c79-ad4c-3ac7bace8da5 None None] MFA Rules not processed for user `7575179edbda45208561c983fc5b956c`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:11.072346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-34bf1738-07b8-4c79-ad4c-3ac7bace8da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:11.129694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1118/2240] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:08:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:21.122825 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=25 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:08:25.331844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016495c0-3589-4631-84c6-145a396e39f4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:25.332448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016495c0-3589-4631-84c6-145a396e39f4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:25.332448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016495c0-3589-4631-84c6-145a396e39f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:25.353785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-016495c0-3589-4631-84c6-145a396e39f4 None None] MFA Rules not processed for user `166b42a5d6c04659afb20c088dfead76`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:25.355021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-016495c0-3589-4631-84c6-145a396e39f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:25.412819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1123/2241] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:08:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:33.356213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240c36f-2d0b-45db-b655-453882ac15cf None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:33.356740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240c36f-2d0b-45db-b655-453882ac15cf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:33.356857 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240c36f-2d0b-45db-b655-453882ac15cf None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:33.358159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1119/2242] 10.222.0.32 () {54 vars in 921 bytes} [Mon May 30 12:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 30 12:08:33.392570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6b9c-4d91-444d-ab2e-d778266b27cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:33.393508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6b9c-4d91-444d-ab2e-d778266b27cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:33.394755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6b9c-4d91-444d-ab2e-d778266b27cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:33.449158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240c36f-2d0b-45db-b655-453882ac15cf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:33.452243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b240c36f-2d0b-45db-b655-453882ac15cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:33.481387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-566f6b9c-4d91-444d-ab2e-d778266b27cb None None] MFA Rules not processed for user `6c654dad1f844fe9995c594116619216`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:33.482713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-566f6b9c-4d91-444d-ab2e-d778266b27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:33.491841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1124/2243] 10.222.0.32 () {58 vars in 1018 bytes} [Mon May 30 12:08:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:33.531943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f6b9c-4d91-444d-ab2e-d778266b27cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:33.533788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-566f6b9c-4d91-444d-ab2e-d778266b27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:33.557555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f6b9c-4d91-444d-ab2e-d778266b27cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:33.565169 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61189515-708a-46b5-9563-3acd076eeeef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:33.566234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189515-708a-46b5-9563-3acd076eeeef service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:33.566646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189515-708a-46b5-9563-3acd076eeeef service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:33.567129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189515-708a-46b5-9563-3acd076eeeef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:33.577206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240c36f-2d0b-45db-b655-453882ac15cf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:33.579075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61189515-708a-46b5-9563-3acd076eeeef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:33.582024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61189515-708a-46b5-9563-3acd076eeeef service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:33.583615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:33.584749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:33.585196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:33.585773 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] PATH_INFO: `/v3/users/1d4c83a1ccfc43beb3f48e4462b06794` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:33.592513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4c83a1ccfc43beb3f48e4462b06794)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:33.594881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:33.624473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1125/2244] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:08:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:08:33.674608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4c83a1ccfc43beb3f48e4462b06794'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:33.688370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4c83a1ccfc43beb3f48e4462b06794'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:33.690877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4c83a1ccfc43beb3f48e4462b06794'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:33.700128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4c83a1ccfc43beb3f48e4462b06794'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:33.701244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb637fc0-8cf6-4d50-8dde-709883118afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4c83a1ccfc43beb3f48e4462b06794'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:33.716511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1120/2245] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:08:33 2022] DELETE /identity/v3/users/1d4c83a1ccfc43beb3f48e4462b06794 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:08:33.924485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61189515-708a-46b5-9563-3acd076eeeef service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:33.943998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61189515-708a-46b5-9563-3acd076eeeef service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:34.033005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:34.034000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.034331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.034601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] PATH_INFO: `/v3/projects/94cd1e0377594f52bb5bfa7159731d48` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.035896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cd1e0377594f52bb5bfa7159731d48)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:34.038174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:34.098523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cd1e0377594f52bb5bfa7159731d48', 'request_id': 'req-cc26a6ad-eb17-4cb2-a662-91809304ae57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:34.110022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cd1e0377594f52bb5bfa7159731d48', 'request_id': 'req-cc26a6ad-eb17-4cb2-a662-91809304ae57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:34.119027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cd1e0377594f52bb5bfa7159731d48', 'request_id': 'req-cc26a6ad-eb17-4cb2-a662-91809304ae57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:34.134045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] The token cache is being invalidate because project 94cd1e0377594f52bb5bfa7159731d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:08:34.134346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:34.135215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc26a6ad-eb17-4cb2-a662-91809304ae57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:34.136836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1126/2246] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:08:33 2022] DELETE /identity/v3/projects/94cd1e0377594f52bb5bfa7159731d48 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:08:34.141817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e455-9154-4376-bb4b-2b054b5107e8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.142125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e455-9154-4376-bb4b-2b054b5107e8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.142406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804e455-9154-4376-bb4b-2b054b5107e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.164801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c804e455-9154-4376-bb4b-2b054b5107e8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:34.166245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c804e455-9154-4376-bb4b-2b054b5107e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:34.242325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1121/2247] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:34.249717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd441d-af63-4070-95f2-b76e96b53b0c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.250124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd441d-af63-4070-95f2-b76e96b53b0c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.250478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd441d-af63-4070-95f2-b76e96b53b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.258436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:08:34.276086 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6ffd441d-af63-4070-95f2-b76e96b53b0c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:34.277465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6ffd441d-af63-4070-95f2-b76e96b53b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:34.286326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1127/2248] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:34.291710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804e455-9154-4376-bb4b-2b054b5107e8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:34.302445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804e455-9154-4376-bb4b-2b054b5107e8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:34.308940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:34.309615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.310711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.310711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.311088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:08:34.311320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:34.313176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:34.327582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1122/2249] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:08:34.331770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd441d-af63-4070-95f2-b76e96b53b0c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:34.343462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd441d-af63-4070-95f2-b76e96b53b0c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:34.352137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c1624-87c7-4c25-997c-88801150b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:34.355452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c1624-87c7-4c25-997c-88801150b24e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.355452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c1624-87c7-4c25-997c-88801150b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.355452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c1624-87c7-4c25-997c-88801150b24e admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.355452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c1624-87c7-4c25-997c-88801150b24e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:34.356351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c1624-87c7-4c25-997c-88801150b24e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:34.392235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1128/2250] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:08:34 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:08:34.396789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:34.410781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ba697-0ac1-44d1-90af-9eb028aa0ae2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:34.417206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:08:34.482336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7cc2c-faf0-4633-b6c1-2a29cd756faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:34.483019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7cc2c-faf0-4633-b6c1-2a29cd756faf admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.483304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7cc2c-faf0-4633-b6c1-2a29cd756faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.483580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7cc2c-faf0-4633-b6c1-2a29cd756faf admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.484748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7cc2c-faf0-4633-b6c1-2a29cd756faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:34.486256 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7cc2c-faf0-4633-b6c1-2a29cd756faf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:34.515630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1123/2251] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:08:34 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:08:34.519852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c1624-87c7-4c25-997c-88801150b24e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:34.533446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c1624-87c7-4c25-997c-88801150b24e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:34.540417 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4051f-d09e-45f1-8df0-87679dbca772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:34.541213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4051f-d09e-45f1-8df0-87679dbca772 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.541575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4051f-d09e-45f1-8df0-87679dbca772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.542750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4051f-d09e-45f1-8df0-87679dbca772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.543584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4051f-d09e-45f1-8df0-87679dbca772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:34.545317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4051f-d09e-45f1-8df0-87679dbca772 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:34.560195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1129/2252] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:08:34.561951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7cc2c-faf0-4633-b6c1-2a29cd756faf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:34.578245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7cc2c-faf0-4633-b6c1-2a29cd756faf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:34.591722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe204f9-429d-40b2-b9ef-88c3d56c1145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:34.592490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe204f9-429d-40b2-b9ef-88c3d56c1145 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.592779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe204f9-429d-40b2-b9ef-88c3d56c1145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.593059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe204f9-429d-40b2-b9ef-88c3d56c1145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.593809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe204f9-429d-40b2-b9ef-88c3d56c1145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:34.595449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe204f9-429d-40b2-b9ef-88c3d56c1145 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:34.616205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1124/2253] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:08:34.619143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4051f-d09e-45f1-8df0-87679dbca772 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:34.630647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4051f-d09e-45f1-8df0-87679dbca772 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:34.636400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:34.637379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.637661 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.637938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] PATH_INFO: `/v3/projects/5cbd0030297e43dca0e4cab9ee7aded4/users/171fab02ec3a41ceac31b2433b084401/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.655811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbd0030297e43dca0e4cab9ee7aded4, user_id=171fab02ec3a41ceac31b2433b084401, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:34.658522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:34.677257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1130/2254] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:08:34 2022] PUT /identity/v3/projects/5cbd0030297e43dca0e4cab9ee7aded4/users/171fab02ec3a41ceac31b2433b084401/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:08:34.684466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c24b6-743a-4276-9a2c-43951f706415 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.684762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c24b6-743a-4276-9a2c-43951f706415 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.685063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c24b6-743a-4276-9a2c-43951f706415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.714212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b59c24b6-743a-4276-9a2c-43951f706415 None None] MFA Rules not processed for user `171fab02ec3a41ceac31b2433b084401`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:34.715443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b59c24b6-743a-4276-9a2c-43951f706415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:34.796680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1125/2255] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:08:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:34.810446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:34.812518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:34.873656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9f973-0fd5-43f3-821e-1f49caf0310a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:34.878699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b615ea-63ba-4bb8-a6b1-cc3d9035a0e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:34.879406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b615ea-63ba-4bb8-a6b1-cc3d9035a0e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:34.879733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b615ea-63ba-4bb8-a6b1-cc3d9035a0e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:34.880017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b615ea-63ba-4bb8-a6b1-cc3d9035a0e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:34.890617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b615ea-63ba-4bb8-a6b1-cc3d9035a0e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:34.892072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b615ea-63ba-4bb8-a6b1-cc3d9035a0e7 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:34.902849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1131/2256] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:08:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:08:35.736087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80c5d0-12c8-4b26-8398-b61d434ef8d3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:35.736602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80c5d0-12c8-4b26-8398-b61d434ef8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:35.736718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80c5d0-12c8-4b26-8398-b61d434ef8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:35.755027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ed80c5d0-12c8-4b26-8398-b61d434ef8d3 None None] MFA Rules not processed for user `bc47c5bf77894aefa6ecde115ed9f88f`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:35.756311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed80c5d0-12c8-4b26-8398-b61d434ef8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:35.813345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1126/2257] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:08:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:44.898791 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:08:45.806103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:08:50.266444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bcfb3-4837-4b39-a8e9-a179694c6f1b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:50.266883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bcfb3-4837-4b39-a8e9-a179694c6f1b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:50.267254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bcfb3-4837-4b39-a8e9-a179694c6f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:50.289697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-443bcfb3-4837-4b39-a8e9-a179694c6f1b None None] MFA Rules not processed for user `c76e8d7bfdc04ea79c83ca7fb6b9ade1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:50.291274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-443bcfb3-4837-4b39-a8e9-a179694c6f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:50.348630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1132/2258] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:08:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:53.344018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80c5d0-12c8-4b26-8398-b61d434ef8d3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:53.418794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80c5d0-12c8-4b26-8398-b61d434ef8d3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:53.425869 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:53.427589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:53.428131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:53.428512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] PATH_INFO: `/v3/users/171fab02ec3a41ceac31b2433b084401` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:53.429944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=171fab02ec3a41ceac31b2433b084401)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:53.432489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:53.532631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '171fab02ec3a41ceac31b2433b084401'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:53.551791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '171fab02ec3a41ceac31b2433b084401'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:53.552791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '171fab02ec3a41ceac31b2433b084401'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:53.560899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '171fab02ec3a41ceac31b2433b084401'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:53.561947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '171fab02ec3a41ceac31b2433b084401'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:53.585152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1127/2259] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:08:53 2022] DELETE /identity/v3/users/171fab02ec3a41ceac31b2433b084401 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:08:53.589541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae627-7806-4995-8555-389bcbd9f5d4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:53.590495 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae627-7806-4995-8555-389bcbd9f5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:53.590992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae627-7806-4995-8555-389bcbd9f5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:53.613596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ca0ae627-7806-4995-8555-389bcbd9f5d4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:08:53.615416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ca0ae627-7806-4995-8555-389bcbd9f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:53.682066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1133/2260] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:08:53.693685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:53.695285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:53.758506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd98819-7e05-4634-bcd8-22666b71e8ea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:53.762199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:53.762860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:53.763146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:53.763427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:53.774402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:53.775863 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:53.787221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1128/2261] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:08:54.054028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ae627-7806-4995-8555-389bcbd9f5d4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:08:54.055473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ca0ae627-7806-4995-8555-389bcbd9f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:08:54.067800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ae627-7806-4995-8555-389bcbd9f5d4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:08:54.072783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:08:54.074621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:08:54.074919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:08:54.075207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] PATH_INFO: `/v3/projects/5cbd0030297e43dca0e4cab9ee7aded4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:08:54.077048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbd0030297e43dca0e4cab9ee7aded4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:08:54.079825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:08:54.134888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbd0030297e43dca0e4cab9ee7aded4', 'request_id': 'req-aa33f770-211c-4b0a-b2d9-3784a15c86b5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:54.145589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbd0030297e43dca0e4cab9ee7aded4', 'request_id': 'req-aa33f770-211c-4b0a-b2d9-3784a15c86b5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:54.155996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbd0030297e43dca0e4cab9ee7aded4', 'request_id': 'req-aa33f770-211c-4b0a-b2d9-3784a15c86b5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:54.167966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] The token cache is being invalidate because project 5cbd0030297e43dca0e4cab9ee7aded4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:08:54.168282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:54.168943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:08:54.170503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1134/2262] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:08:54 2022] DELETE /identity/v3/projects/5cbd0030297e43dca0e4cab9ee7aded4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:09:03.782044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:04.164616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:09.819959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:09.821395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:09.877294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad05059-4303-4662-b4d4-38deb8d8cba0 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:09.926385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:09.927397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:09.927696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:09.927958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:09.929826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:10.000504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:10.001954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:10.014758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1129/2263] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:13.371624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:13.373048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:13.385751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33f770-211c-4b0a-b2d9-3784a15c86b5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:13.389339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:13.389981 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:13.390269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:13.390550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:13.392399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:13.407499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:13.408975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:13.419847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1135/2264] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:18.236374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:18.245616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf11934-85c9-40b9-8c6e-335355079fd9 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:18.249924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:18.250523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:18.250826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:18.251088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:18.252966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:18.311996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:18.315264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:18.317156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:18.326805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1130/2265] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:23.415097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:27.496891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:27.498356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:27.512006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face0ec3-cc5e-4d88-bc93-806c352a23db service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:27.517548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e064cf8-0e8a-4da2-813b-a571d773357c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:27.518186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064cf8-0e8a-4da2-813b-a571d773357c service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:27.518503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064cf8-0e8a-4da2-813b-a571d773357c service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:27.519070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e064cf8-0e8a-4da2-813b-a571d773357c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:27.520900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7e064cf8-0e8a-4da2-813b-a571d773357c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:27.579425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e064cf8-0e8a-4da2-813b-a571d773357c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:27.581103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e064cf8-0e8a-4da2-813b-a571d773357c service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:27.590280 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1136/2266] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:28.322317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:32.278807 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:32.280344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:32.346345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dc37f-7bbe-46f7-8554-26966f819155 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:32.350152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:32.350815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:32.351105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:32.351388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:32.353232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:32.431305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:32.432904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:32.444562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1131/2267] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:34.508927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b7032-c644-4727-8b1c-7e2a6ef12c95 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:34.509797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b7032-c644-4727-8b1c-7e2a6ef12c95 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:34.510074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b7032-c644-4727-8b1c-7e2a6ef12c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:34.528485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-740b7032-c644-4727-8b1c-7e2a6ef12c95 None None] MFA Rules not processed for user `7575179edbda45208561c983fc5b956c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:09:34.529706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-740b7032-c644-4727-8b1c-7e2a6ef12c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:34.538593 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1137/2268] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:09:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:09:35.061596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:35.071392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3afafc-19cb-4198-806a-94f879e6305c service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:35.075409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:35.076064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:35.076372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:35.076648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:35.078465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:35.091685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:35.093199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:35.103427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1132/2269] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:35.575761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b7032-c644-4727-8b1c-7e2a6ef12c95 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:35.578549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-740b7032-c644-4727-8b1c-7e2a6ef12c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:35.702175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b7032-c644-4727-8b1c-7e2a6ef12c95 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:35.709523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:35.710319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:35.710521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:35.710984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:35.713154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:35.727473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:35.732604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:35.750258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1138/2270] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:36.361291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:36.364373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:36.386537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b205ea3-5a07-4d68-a559-cad8d4bdc816 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:36.394245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:36.395259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:36.395813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:36.396186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] PATH_INFO: `/v3/users/166b42a5d6c04659afb20c088dfead76` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:36.397485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166b42a5d6c04659afb20c088dfead76)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:36.399453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:36.462268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166b42a5d6c04659afb20c088dfead76'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.476377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166b42a5d6c04659afb20c088dfead76'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.477162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166b42a5d6c04659afb20c088dfead76'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.484574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166b42a5d6c04659afb20c088dfead76'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.485324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2635cdf0-c02e-4155-9180-eafcc89565d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166b42a5d6c04659afb20c088dfead76'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.500986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1133/2271] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:09:36 2022] DELETE /identity/v3/users/166b42a5d6c04659afb20c088dfead76 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:09:36.703543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:36.716599 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699d838-bd02-4bfd-9772-797b3aed68c4 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:36.779292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:36.780177 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:36.780531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:36.780806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] PATH_INFO: `/v3/projects/c988db8e79bf4c98ac288105177f90f2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:36.781996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c988db8e79bf4c98ac288105177f90f2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:36.784054 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:36.812286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:36.838692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c988db8e79bf4c98ac288105177f90f2', 'request_id': 'req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.849340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c988db8e79bf4c98ac288105177f90f2', 'request_id': 'req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.857544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c988db8e79bf4c98ac288105177f90f2', 'request_id': 'req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.869362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] The token cache is being invalidate because project c988db8e79bf4c98ac288105177f90f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:09:36.869657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.870396 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa578a1e-672d-43d9-b4ac-5afd0d6b57d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:09:36.872306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1139/2272] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:09:36 2022] DELETE /identity/v3/projects/c988db8e79bf4c98ac288105177f90f2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:09:36.878973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2b6c0-0ea9-4db6-83bb-394eae2af47b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:36.879458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2b6c0-0ea9-4db6-83bb-394eae2af47b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:36.879881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2b6c0-0ea9-4db6-83bb-394eae2af47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:36.900255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-65c2b6c0-0ea9-4db6-83bb-394eae2af47b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:09:36.901663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-65c2b6c0-0ea9-4db6-83bb-394eae2af47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:36.906740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:37.005984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1134/2273] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:09:37.013529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57f3e7-404b-4fe5-b4ec-f8d50336c899 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.013964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57f3e7-404b-4fe5-b4ec-f8d50336c899 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.014340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57f3e7-404b-4fe5-b4ec-f8d50336c899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.041313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-af57f3e7-404b-4fe5-b4ec-f8d50336c899 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:09:37.045545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af57f3e7-404b-4fe5-b4ec-f8d50336c899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:37.059785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1140/2274] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:09:37.060313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2b6c0-0ea9-4db6-83bb-394eae2af47b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.071148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2b6c0-0ea9-4db6-83bb-394eae2af47b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.076194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.076955 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.077257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.077534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.078398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:09:37.078722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.080370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.093125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1135/2275] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:09:37.098047 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57f3e7-404b-4fe5-b4ec-f8d50336c899 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.107134 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57f3e7-404b-4fe5-b4ec-f8d50336c899 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.111746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb9f5-da08-4a5b-b85f-df1a90b57304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.112449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb9f5-da08-4a5b-b85f-df1a90b57304 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.112735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb9f5-da08-4a5b-b85f-df1a90b57304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.113015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb9f5-da08-4a5b-b85f-df1a90b57304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.113714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cb9f5-da08-4a5b-b85f-df1a90b57304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.115627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cb9f5-da08-4a5b-b85f-df1a90b57304 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.140862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1141/2276] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:09:37 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:09:37.146326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.156646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e066add-09d2-477b-8aa4-dc059e258add admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.216825 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4b9a7-8991-42a6-acff-004584e869ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.217496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4b9a7-8991-42a6-acff-004584e869ce admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.217789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4b9a7-8991-42a6-acff-004584e869ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.218068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4b9a7-8991-42a6-acff-004584e869ce admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.218782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4b9a7-8991-42a6-acff-004584e869ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.220856 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4b9a7-8991-42a6-acff-004584e869ce admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.247734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1136/2277] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:09:37 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:09:37.249321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb9f5-da08-4a5b-b85f-df1a90b57304 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.260813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb9f5-da08-4a5b-b85f-df1a90b57304 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.265913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ec4f6-cd9b-429e-9b47-2aad09255adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.266591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ec4f6-cd9b-429e-9b47-2aad09255adf admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.266881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ec4f6-cd9b-429e-9b47-2aad09255adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.267162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ec4f6-cd9b-429e-9b47-2aad09255adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.267898 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ec4f6-cd9b-429e-9b47-2aad09255adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.270189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ec4f6-cd9b-429e-9b47-2aad09255adf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.281226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1142/2278] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:09:37.285358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4b9a7-8991-42a6-acff-004584e869ce admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.296672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4b9a7-8991-42a6-acff-004584e869ce admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.301911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c560700-189d-4309-a984-aad14f3bdc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.302590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560700-189d-4309-a984-aad14f3bdc40 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.302877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560700-189d-4309-a984-aad14f3bdc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.303155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560700-189d-4309-a984-aad14f3bdc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.303887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c560700-189d-4309-a984-aad14f3bdc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.305541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c560700-189d-4309-a984-aad14f3bdc40 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.316925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1137/2279] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:09:37.321105 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ec4f6-cd9b-429e-9b47-2aad09255adf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.331122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ec4f6-cd9b-429e-9b47-2aad09255adf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.335860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ca928-6824-4a64-a7ca-7fe9818ce35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.336781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca928-6824-4a64-a7ca-7fe9818ce35c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.337062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca928-6824-4a64-a7ca-7fe9818ce35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.337337 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca928-6824-4a64-a7ca-7fe9818ce35c admin admin] PATH_INFO: `/v3/projects/3111bc69299a429d92f5194903da130b/users/05a779ec90884cbca4921d0325f8b518/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.354227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ca928-6824-4a64-a7ca-7fe9818ce35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3111bc69299a429d92f5194903da130b, user_id=05a779ec90884cbca4921d0325f8b518, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.356836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ca928-6824-4a64-a7ca-7fe9818ce35c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.372716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1143/2280] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:09:37 2022] PUT /identity/v3/projects/3111bc69299a429d92f5194903da130b/users/05a779ec90884cbca4921d0325f8b518/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:09:37.384049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cdf89-9ed5-4207-8a7d-cf136fd7e905 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.384777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cdf89-9ed5-4207-8a7d-cf136fd7e905 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.384777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cdf89-9ed5-4207-8a7d-cf136fd7e905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.413847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8a7cdf89-9ed5-4207-8a7d-cf136fd7e905 None None] MFA Rules not processed for user `05a779ec90884cbca4921d0325f8b518`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:09:37.415095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a7cdf89-9ed5-4207-8a7d-cf136fd7e905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:37.482082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1138/2281] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:09:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:09:37.486284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ca928-6824-4a64-a7ca-7fe9818ce35c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.551584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ca928-6824-4a64-a7ca-7fe9818ce35c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.556369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e0b68-5da2-426e-875f-8df01637e7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.557033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e0b68-5da2-426e-875f-8df01637e7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.557316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e0b68-5da2-426e-875f-8df01637e7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.557592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e0b68-5da2-426e-875f-8df01637e7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.558300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e0b68-5da2-426e-875f-8df01637e7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.560200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e0b68-5da2-426e-875f-8df01637e7a9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.590048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1144/2282] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:09:37 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:09:37.594847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cdf89-9ed5-4207-8a7d-cf136fd7e905 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.601781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:37.607053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cdf89-9ed5-4207-8a7d-cf136fd7e905 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.682838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.683827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.684203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.684559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.685378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.687785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.715589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1139/2283] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:09:37 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:09:37.719327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e0b68-5da2-426e-875f-8df01637e7a9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.731700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.733423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:37.735294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e0b68-5da2-426e-875f-8df01637e7a9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.742443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194832c-9499-48df-9c9c-9e5f094dabb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.743470 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194832c-9499-48df-9c9c-9e5f094dabb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.744006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194832c-9499-48df-9c9c-9e5f094dabb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.744521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194832c-9499-48df-9c9c-9e5f094dabb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.745550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194832c-9499-48df-9c9c-9e5f094dabb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.747452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194832c-9499-48df-9c9c-9e5f094dabb4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.767552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1145/2284] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:09:37.769893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194832c-9499-48df-9c9c-9e5f094dabb4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.796621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194832c-9499-48df-9c9c-9e5f094dabb4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.810121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed372d39-bb11-4b72-b797-6d4180adbc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.811496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed372d39-bb11-4b72-b797-6d4180adbc21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.812009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed372d39-bb11-4b72-b797-6d4180adbc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.812464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed372d39-bb11-4b72-b797-6d4180adbc21 admin admin] PATH_INFO: `/v3/projects/8a80c2582a864ede8a8d02f87557126a/users/424fd798cf2e44dfb59884bfe3d12bfb/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.835872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed372d39-bb11-4b72-b797-6d4180adbc21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a80c2582a864ede8a8d02f87557126a, user_id=424fd798cf2e44dfb59884bfe3d12bfb, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.839320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed372d39-bb11-4b72-b797-6d4180adbc21 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.852741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:37.858505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb6dd3-71ea-44cb-8373-def74dcf0339 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.867374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1146/2285] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:09:37 2022] PUT /identity/v3/projects/8a80c2582a864ede8a8d02f87557126a/users/424fd798cf2e44dfb59884bfe3d12bfb/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:09:37.871418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed372d39-bb11-4b72-b797-6d4180adbc21 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.929995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.930659 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.930945 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.931229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.934290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:37.945759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.946222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed372d39-bb11-4b72-b797-6d4180adbc21 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:37.947228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.951798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d788dcd-288a-4304-b926-00db5e6d54fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:37.952845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d788dcd-288a-4304-b926-00db5e6d54fc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:37.953429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d788dcd-288a-4304-b926-00db5e6d54fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:37.954488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d788dcd-288a-4304-b926-00db5e6d54fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:37.955544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d788dcd-288a-4304-b926-00db5e6d54fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:37.957921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d788dcd-288a-4304-b926-00db5e6d54fc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:37.960348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1140/2286] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:37.970234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1147/2287] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:09:37.974919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:37.989774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606f14-b0d7-46c2-a8b9-15f0abbb5153 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:38.005735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:38.007515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:38.008067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:38.008851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] PATH_INFO: `/v3/projects/8a80c2582a864ede8a8d02f87557126a/users/424fd798cf2e44dfb59884bfe3d12bfb/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:38.011588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a80c2582a864ede8a8d02f87557126a, user_id=424fd798cf2e44dfb59884bfe3d12bfb, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:38.014672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:38.027391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1141/2288] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:09:37 2022] PUT /identity/v3/projects/8a80c2582a864ede8a8d02f87557126a/users/424fd798cf2e44dfb59884bfe3d12bfb/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:09:38.038377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190022da-2780-4c37-adfa-d16f06902770 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:38.038679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190022da-2780-4c37-adfa-d16f06902770 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:38.038974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190022da-2780-4c37-adfa-d16f06902770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:38.067275 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-190022da-2780-4c37-adfa-d16f06902770 None None] MFA Rules not processed for user `424fd798cf2e44dfb59884bfe3d12bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:09:38.068934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-190022da-2780-4c37-adfa-d16f06902770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:38.140210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1148/2289] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:09:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:09:38.152406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:38.154141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:38.204090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:38.221429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0a23-e0f4-4b68-ba1a-46c1f4b1aa2c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:38.225430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:38.226117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:38.226402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:38.226708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:38.236264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:38.238430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:38.248467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1142/2290] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:38.329375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190022da-2780-4c37-adfa-d16f06902770 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:38.341067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190022da-2780-4c37-adfa-d16f06902770 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:38.345218 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff91f-ee4e-421a-bc50-8fb21a0e9811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:38.345961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff91f-ee4e-421a-bc50-8fb21a0e9811 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:38.346390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff91f-ee4e-421a-bc50-8fb21a0e9811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:38.346773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ff91f-ee4e-421a-bc50-8fb21a0e9811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:38.410025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ff91f-ee4e-421a-bc50-8fb21a0e9811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:38.411673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ff91f-ee4e-421a-bc50-8fb21a0e9811 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:38.422961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1149/2291] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:38.671474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:38.673525 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:38.771631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b8312-cf6e-473a-9a62-4fdf6222f579 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:38.777367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:38.778094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:38.778380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:38.778663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:38.780681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:38.798489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:38.800693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:38.813364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1143/2292] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:39.845525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff91f-ee4e-421a-bc50-8fb21a0e9811 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:39.859596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ff91f-ee4e-421a-bc50-8fb21a0e9811 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:39.870800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:39.871581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:39.872145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:39.872484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:39.874519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:39.892034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:39.972437 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:39.984309 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:39.997382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1150/2293] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:45.667095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:45.668528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:45.682971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa78b29-bd74-4e0d-8e73-074891efb988 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:45.688514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:45.689152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:45.689432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:45.689790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:45.691926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:45.752055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:45.753702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:45.762748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1144/2294] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:47.692884 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:09:47.707893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3c227-451f-486c-afe1-fd652ed58e09 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:09:47.718930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9fbfc-1e3d-4d5d-b6f6-861389d78029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:09:47.719849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9fbfc-1e3d-4d5d-b6f6-861389d78029 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:09:47.720448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9fbfc-1e3d-4d5d-b6f6-861389d78029 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:09:47.721678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9fbfc-1e3d-4d5d-b6f6-861389d78029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:09:47.724022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-01a9fbfc-1e3d-4d5d-b6f6-861389d78029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:09:47.742031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9fbfc-1e3d-4d5d-b6f6-861389d78029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:09:47.744106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9fbfc-1e3d-4d5d-b6f6-861389d78029 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:09:47.761446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1151/2295] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:09:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:09:55.758523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=70 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:09:57.755921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=26 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:06.139567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:06.154501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dc8ce-1058-4eb9-b6fc-8994596bfa77 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:06.160337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:06.161176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:06.161322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:06.161605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:06.163688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:06.176614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:06.178965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:06.191330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1145/2296] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:08.342690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076519-7cf3-4834-9404-91bf4f2742a7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:08.342690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076519-7cf3-4834-9404-91bf4f2742a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:08.343161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076519-7cf3-4834-9404-91bf4f2742a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:08.362050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d3076519-7cf3-4834-9404-91bf4f2742a7 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:08.363266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d3076519-7cf3-4834-9404-91bf4f2742a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:08.422122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1152/2297] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:10:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:08.522858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:08.525075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:08.633720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70a09a-40d6-4acc-a5fe-bd90236faaa6 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:08.639119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0384ccc-aae8-42a2-98d4-bdf06d718782 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:08.639779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0384ccc-aae8-42a2-98d4-bdf06d718782 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:08.640067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0384ccc-aae8-42a2-98d4-bdf06d718782 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:08.640398 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0384ccc-aae8-42a2-98d4-bdf06d718782 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:08.649939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0384ccc-aae8-42a2-98d4-bdf06d718782 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:08.651535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0384ccc-aae8-42a2-98d4-bdf06d718782 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:08.659952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1146/2298] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:10:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:10:18.415278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=25 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:18.656788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=38 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:19.803131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076519-7cf3-4834-9404-91bf4f2742a7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:19.877016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076519-7cf3-4834-9404-91bf4f2742a7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:19.882792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:19.883672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:19.883955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:19.884261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] PATH_INFO: `/v3/users/05a779ec90884cbca4921d0325f8b518` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:19.885607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a779ec90884cbca4921d0325f8b518)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:19.887559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:19.942836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a779ec90884cbca4921d0325f8b518'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:19.955052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a779ec90884cbca4921d0325f8b518'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:19.955739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a779ec90884cbca4921d0325f8b518'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:19.956405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a779ec90884cbca4921d0325f8b518'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:19.964188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a779ec90884cbca4921d0325f8b518'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:19.977713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1153/2299] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:19 2022] DELETE /identity/v3/users/05a779ec90884cbca4921d0325f8b518 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:19.982205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4522e-1a70-4c26-82de-0d16cd8fd24f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:19.982676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4522e-1a70-4c26-82de-0d16cd8fd24f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:19.982814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4522e-1a70-4c26-82de-0d16cd8fd24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:20.000705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-84f4522e-1a70-4c26-82de-0d16cd8fd24f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:20.001954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-84f4522e-1a70-4c26-82de-0d16cd8fd24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:20.065872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1147/2300] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:20.075912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:20.077365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:20.139186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226313a-19cc-4264-bd81-2f62010de84b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:20.142856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b808ab-6888-497a-9ebb-40973b6244aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:20.143759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b808ab-6888-497a-9ebb-40973b6244aa service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:20.144046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b808ab-6888-497a-9ebb-40973b6244aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:20.144358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b808ab-6888-497a-9ebb-40973b6244aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:20.153953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b808ab-6888-497a-9ebb-40973b6244aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:20.155384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b808ab-6888-497a-9ebb-40973b6244aa service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:20.164852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1154/2301] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:20.471612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4522e-1a70-4c26-82de-0d16cd8fd24f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:20.474047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-84f4522e-1a70-4c26-82de-0d16cd8fd24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:20.491003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4522e-1a70-4c26-82de-0d16cd8fd24f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:20.495814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:20.496715 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:20.497026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:20.497299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] PATH_INFO: `/v3/projects/3111bc69299a429d92f5194903da130b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:20.498582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3111bc69299a429d92f5194903da130b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:20.500401 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:20.549931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3111bc69299a429d92f5194903da130b', 'request_id': 'req-4103ec1e-9bc0-4c1c-abe0-c595c2080080'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.558090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3111bc69299a429d92f5194903da130b', 'request_id': 'req-4103ec1e-9bc0-4c1c-abe0-c595c2080080'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.563899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3111bc69299a429d92f5194903da130b', 'request_id': 'req-4103ec1e-9bc0-4c1c-abe0-c595c2080080'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.581037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] The token cache is being invalidate because project 3111bc69299a429d92f5194903da130b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:20.581326 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.581963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.584786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1148/2302] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:20 2022] DELETE /identity/v3/projects/3111bc69299a429d92f5194903da130b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:20.588985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b808ab-6888-497a-9ebb-40973b6244aa service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:20.599137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b808ab-6888-497a-9ebb-40973b6244aa service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:20.618180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:20.661327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:20.662587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:20.662587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:20.662785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] PATH_INFO: `/v3/users/424fd798cf2e44dfb59884bfe3d12bfb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:20.663994 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=424fd798cf2e44dfb59884bfe3d12bfb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:20.665988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:20.715004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '424fd798cf2e44dfb59884bfe3d12bfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.726590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '424fd798cf2e44dfb59884bfe3d12bfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.727259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '424fd798cf2e44dfb59884bfe3d12bfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.727868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '424fd798cf2e44dfb59884bfe3d12bfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.734626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-94e034c3-3408-4e87-8b3e-394555b9be8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '424fd798cf2e44dfb59884bfe3d12bfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:20.747831 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1155/2303] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:20 2022] DELETE /identity/v3/users/424fd798cf2e44dfb59884bfe3d12bfb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:21.368017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:21.369897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:21.386411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103ec1e-9bc0-4c1c-abe0-c595c2080080 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:21.459377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:21.460280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:21.460564 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:21.472404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] PATH_INFO: `/v3/projects/8a80c2582a864ede8a8d02f87557126a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:21.475442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a80c2582a864ede8a8d02f87557126a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:21.479469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:21.517106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:21.542703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a80c2582a864ede8a8d02f87557126a', 'request_id': 'req-18662810-090e-4587-bcf0-9fae2c3f34d5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:21.555633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a80c2582a864ede8a8d02f87557126a', 'request_id': 'req-18662810-090e-4587-bcf0-9fae2c3f34d5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:21.561616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a80c2582a864ede8a8d02f87557126a', 'request_id': 'req-18662810-090e-4587-bcf0-9fae2c3f34d5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:21.576783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] The token cache is being invalidate because project 8a80c2582a864ede8a8d02f87557126a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:21.577158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:21.577707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-18662810-090e-4587-bcf0-9fae2c3f34d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:21.579274 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1149/2304] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:21 2022] DELETE /identity/v3/projects/8a80c2582a864ede8a8d02f87557126a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:21.584863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6965c7-bd2b-433a-9187-5f8c452670d6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:21.585331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6965c7-bd2b-433a-9187-5f8c452670d6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:21.588287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6965c7-bd2b-433a-9187-5f8c452670d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:21.607828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1a6965c7-bd2b-433a-9187-5f8c452670d6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:21.609097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1a6965c7-bd2b-433a-9187-5f8c452670d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:21.675044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1156/2305] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:21.684292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581139-e256-4b6d-bf04-841a3a49c0d6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:21.684584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581139-e256-4b6d-bf04-841a3a49c0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:21.684868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b581139-e256-4b6d-bf04-841a3a49c0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:21.705610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8b581139-e256-4b6d-bf04-841a3a49c0d6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:21.706827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8b581139-e256-4b6d-bf04-841a3a49c0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:21.715571 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1150/2306] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:21.720046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6965c7-bd2b-433a-9187-5f8c452670d6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:21.730837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6965c7-bd2b-433a-9187-5f8c452670d6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:21.735487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:21.736143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:21.736454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:21.736733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:21.737455 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:10:21.737762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:21.739682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:21.751776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1157/2307] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:10:21.756303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b581139-e256-4b6d-bf04-841a3a49c0d6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:21.765847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b581139-e256-4b6d-bf04-841a3a49c0d6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:21.771540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec0d1a-8f97-4947-89ca-0d08c3511280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:21.772230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec0d1a-8f97-4947-89ca-0d08c3511280 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:21.772524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec0d1a-8f97-4947-89ca-0d08c3511280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:21.772806 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec0d1a-8f97-4947-89ca-0d08c3511280 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:21.773508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec0d1a-8f97-4947-89ca-0d08c3511280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:21.775455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec0d1a-8f97-4947-89ca-0d08c3511280 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:21.815787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1151/2308] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:21 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:21.816927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:21.827521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fa3c1-4c35-43ef-9ace-84d61e59d0cd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:21.855790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:21.890149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4cab7-1631-4ff7-971a-5be7cd6091b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:21.892182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4cab7-1631-4ff7-971a-5be7cd6091b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:21.892482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4cab7-1631-4ff7-971a-5be7cd6091b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:21.892759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4cab7-1631-4ff7-971a-5be7cd6091b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:21.894357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4cab7-1631-4ff7-971a-5be7cd6091b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:21.898411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4cab7-1631-4ff7-971a-5be7cd6091b6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:21.920230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1158/2309] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:21 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:21.924329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec0d1a-8f97-4947-89ca-0d08c3511280 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:21.934975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec0d1a-8f97-4947-89ca-0d08c3511280 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:21.939707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929e6fe-d723-4978-9dff-2069b9d165aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:21.940403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929e6fe-d723-4978-9dff-2069b9d165aa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:21.940689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929e6fe-d723-4978-9dff-2069b9d165aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:21.940963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929e6fe-d723-4978-9dff-2069b9d165aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:21.941750 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929e6fe-d723-4978-9dff-2069b9d165aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:21.943464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929e6fe-d723-4978-9dff-2069b9d165aa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:21.953390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1152/2310] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:21.958143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4cab7-1631-4ff7-971a-5be7cd6091b6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:21.968987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4cab7-1631-4ff7-971a-5be7cd6091b6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:21.974606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdd623-7d17-40e0-84eb-8a81b5d1747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:21.975325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdd623-7d17-40e0-84eb-8a81b5d1747d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:21.975610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdd623-7d17-40e0-84eb-8a81b5d1747d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:21.975886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdd623-7d17-40e0-84eb-8a81b5d1747d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:21.976660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdd623-7d17-40e0-84eb-8a81b5d1747d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:21.978611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdd623-7d17-40e0-84eb-8a81b5d1747d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:21.989543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1159/2311] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:10:21.993749 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929e6fe-d723-4978-9dff-2069b9d165aa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.003718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929e6fe-d723-4978-9dff-2069b9d165aa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.009025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d3ee6-04a7-466a-a77f-8b8b3e439018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.009909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d3ee6-04a7-466a-a77f-8b8b3e439018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.010192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d3ee6-04a7-466a-a77f-8b8b3e439018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.010467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d3ee6-04a7-466a-a77f-8b8b3e439018 admin admin] PATH_INFO: `/v3/projects/7b75d7c218d44df58d1d0f41ea0bc10b/users/c3bf1694d0284d3d900104304c8f07fd/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.026284 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d3ee6-04a7-466a-a77f-8b8b3e439018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b75d7c218d44df58d1d0f41ea0bc10b, user_id=c3bf1694d0284d3d900104304c8f07fd, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:22.028989 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d3ee6-04a7-466a-a77f-8b8b3e439018 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:22.043170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1153/2312] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:21 2022] PUT /identity/v3/projects/7b75d7c218d44df58d1d0f41ea0bc10b/users/c3bf1694d0284d3d900104304c8f07fd/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:10:22.053513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedea33-158a-4a50-b28a-ac91dd02b931 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.053808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedea33-158a-4a50-b28a-ac91dd02b931 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.054087 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccedea33-158a-4a50-b28a-ac91dd02b931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.080535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ccedea33-158a-4a50-b28a-ac91dd02b931 None None] MFA Rules not processed for user `c3bf1694d0284d3d900104304c8f07fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:22.081663 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ccedea33-158a-4a50-b28a-ac91dd02b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:22.145980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1160/2313] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:22.150449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d3ee6-04a7-466a-a77f-8b8b3e439018 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.212007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d3ee6-04a7-466a-a77f-8b8b3e439018 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.216742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034d9e3-fa4a-482b-ae55-bf31c0071102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.217406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034d9e3-fa4a-482b-ae55-bf31c0071102 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.217690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034d9e3-fa4a-482b-ae55-bf31c0071102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.217974 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034d9e3-fa4a-482b-ae55-bf31c0071102 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.218660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034d9e3-fa4a-482b-ae55-bf31c0071102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:22.220612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034d9e3-fa4a-482b-ae55-bf31c0071102 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:22.245510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1154/2314] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:22 2022] POST /identity/v3/projects => generated 361 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:22.249532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedea33-158a-4a50-b28a-ac91dd02b931 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.261426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccedea33-158a-4a50-b28a-ac91dd02b931 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.329440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c7dbb-9b7f-47f8-8123-a39b226ac461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.330120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c7dbb-9b7f-47f8-8123-a39b226ac461 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.330406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c7dbb-9b7f-47f8-8123-a39b226ac461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.330741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c7dbb-9b7f-47f8-8123-a39b226ac461 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.331389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c7dbb-9b7f-47f8-8123-a39b226ac461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:22.334038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c7dbb-9b7f-47f8-8123-a39b226ac461 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:22.353414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1161/2315] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:22 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:22.357406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034d9e3-fa4a-482b-ae55-bf31c0071102 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.367962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034d9e3-fa4a-482b-ae55-bf31c0071102 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.373896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314cc24-386a-4950-9c3d-e07c80d181ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.374569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cc24-386a-4950-9c3d-e07c80d181ec admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.374876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cc24-386a-4950-9c3d-e07c80d181ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.375108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cc24-386a-4950-9c3d-e07c80d181ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.375841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314cc24-386a-4950-9c3d-e07c80d181ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:22.377495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314cc24-386a-4950-9c3d-e07c80d181ec admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:22.389574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1155/2316] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:22.393996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c7dbb-9b7f-47f8-8123-a39b226ac461 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.408919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c7dbb-9b7f-47f8-8123-a39b226ac461 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.415623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39dab0-f095-469e-8d51-123008538ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.416923 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39dab0-f095-469e-8d51-123008538ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.417436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39dab0-f095-469e-8d51-123008538ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.417878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39dab0-f095-469e-8d51-123008538ed6 admin admin] PATH_INFO: `/v3/projects/9ebd2be4cfc3429c9a49efae8cdabedd/users/2545ccb29e7b45d0b66f3c2558113f6d/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.436852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39dab0-f095-469e-8d51-123008538ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2be4cfc3429c9a49efae8cdabedd, user_id=2545ccb29e7b45d0b66f3c2558113f6d, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:22.440109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39dab0-f095-469e-8d51-123008538ed6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:22.455188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1162/2317] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:22 2022] PUT /identity/v3/projects/9ebd2be4cfc3429c9a49efae8cdabedd/users/2545ccb29e7b45d0b66f3c2558113f6d/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:22.460605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314cc24-386a-4950-9c3d-e07c80d181ec admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.472862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314cc24-386a-4950-9c3d-e07c80d181ec admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.487095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:22.547043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069fbbe-d035-42c0-b3e1-4ccdcdd8af90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.547726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069fbbe-d035-42c0-b3e1-4ccdcdd8af90 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.548009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069fbbe-d035-42c0-b3e1-4ccdcdd8af90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.548318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069fbbe-d035-42c0-b3e1-4ccdcdd8af90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.549060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069fbbe-d035-42c0-b3e1-4ccdcdd8af90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:22.550708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069fbbe-d035-42c0-b3e1-4ccdcdd8af90 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:22.566470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1156/2318] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:22.569746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39dab0-f095-469e-8d51-123008538ed6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.578106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:22.587195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39dab0-f095-469e-8d51-123008538ed6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.597565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.598868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.599164 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.599440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] PATH_INFO: `/v3/projects/9ebd2be4cfc3429c9a49efae8cdabedd/users/2545ccb29e7b45d0b66f3c2558113f6d/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.601667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2be4cfc3429c9a49efae8cdabedd, user_id=2545ccb29e7b45d0b66f3c2558113f6d, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:22.605621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:22.618813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1163/2319] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:22 2022] PUT /identity/v3/projects/9ebd2be4cfc3429c9a49efae8cdabedd/users/2545ccb29e7b45d0b66f3c2558113f6d/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:10:22.631733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cc125-0be0-4c8a-a6c1-27bc20d295f4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.632272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cc125-0be0-4c8a-a6c1-27bc20d295f4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.632635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cc125-0be0-4c8a-a6c1-27bc20d295f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.665949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ca7cc125-0be0-4c8a-a6c1-27bc20d295f4 None None] MFA Rules not processed for user `2545ccb29e7b45d0b66f3c2558113f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:22.667286 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ca7cc125-0be0-4c8a-a6c1-27bc20d295f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:22.765377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1157/2320] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:22 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:22.778473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.782505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:22.839979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cc125-0be0-4c8a-a6c1-27bc20d295f4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:22.842485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ca7cc125-0be0-4c8a-a6c1-27bc20d295f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:22.867301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cc4d1-91df-4015-9399-afe10d7eea3b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.871555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc39af-5797-4c22-8d85-682fcb62bd71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.872259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc39af-5797-4c22-8d85-682fcb62bd71 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.872553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc39af-5797-4c22-8d85-682fcb62bd71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.872839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc39af-5797-4c22-8d85-682fcb62bd71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.887187 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc39af-5797-4c22-8d85-682fcb62bd71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:22.888957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc39af-5797-4c22-8d85-682fcb62bd71 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:22.904056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1164/2321] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:22 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:22.933145 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cc125-0be0-4c8a-a6c1-27bc20d295f4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:22.937858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188514e-abff-4c85-b198-68a8df2b4ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:22.938665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188514e-abff-4c85-b198-68a8df2b4ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:22.938953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188514e-abff-4c85-b198-68a8df2b4ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:22.939240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188514e-abff-4c85-b198-68a8df2b4ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:22.942779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0188514e-abff-4c85-b198-68a8df2b4ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:23.025651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc39af-5797-4c22-8d85-682fcb62bd71 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:23.025651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc39af-5797-4c22-8d85-682fcb62bd71 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:23.029235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:23.030085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:23.030447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:23.030859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:23.031442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:23.055803 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188514e-abff-4c85-b198-68a8df2b4ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:23.059102 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188514e-abff-4c85-b198-68a8df2b4ef0 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:23.074351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1158/2322] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:10:23.109538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:23.111521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:23.126053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1165/2323] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:26.216739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e12d3f-e8c5-472a-8a62-483b41a3ef82 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:26.217301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e12d3f-e8c5-472a-8a62-483b41a3ef82 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:26.217301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e12d3f-e8c5-472a-8a62-483b41a3ef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:26.239096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-76e12d3f-e8c5-472a-8a62-483b41a3ef82 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:26.245881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-76e12d3f-e8c5-472a-8a62-483b41a3ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:26.333534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1159/2324] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:10:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:26.351778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:26.354067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:26.459260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a2c00-46d1-4f7c-817b-a50658bc880e service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:26.466104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:26.466913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:26.467270 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:26.467641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:26.485264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:26.486937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:26.497894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1166/2325] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:10:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:10:27.494572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e12d3f-e8c5-472a-8a62-483b41a3ef82 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:27.496380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-76e12d3f-e8c5-472a-8a62-483b41a3ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:27.595294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e12d3f-e8c5-472a-8a62-483b41a3ef82 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:27.601596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:27.602270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:27.602556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:27.603010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:27.605465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:27.614466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:27.616646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:27.624512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:27.626332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:27.641983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48dba42-aa7d-4efb-b248-9e0ca9ebd3af service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:27.656984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:27.657888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:27.658166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:27.658465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] PATH_INFO: `/v3/users/c76e8d7bfdc04ea79c83ca7fb6b9ade1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:27.659658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76e8d7bfdc04ea79c83ca7fb6b9ade1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:27.662025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:27.662169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1160/2326] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:27.710915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:27.751921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76e8d7bfdc04ea79c83ca7fb6b9ade1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:27.771303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76e8d7bfdc04ea79c83ca7fb6b9ade1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:27.772458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76e8d7bfdc04ea79c83ca7fb6b9ade1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:27.773274 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76e8d7bfdc04ea79c83ca7fb6b9ade1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:27.783471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4f955a1f-9e74-4e11-94fd-75bfd57d65dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76e8d7bfdc04ea79c83ca7fb6b9ade1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:27.808243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1167/2327] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:27 2022] DELETE /identity/v3/users/c76e8d7bfdc04ea79c83ca7fb6b9ade1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:10:28.031051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:28.042675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea17eb-bacc-4652-9aa2-eebcb1515680 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:28.122022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:28.122910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.123200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.123481 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] PATH_INFO: `/v3/projects/8b5074cf3e6248668995f266845c8969` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.124722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5074cf3e6248668995f266845c8969)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:28.126747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:28.153235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:28.175382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5074cf3e6248668995f266845c8969', 'request_id': 'req-d72fa229-d786-4ae4-8b37-27da61058e00'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:28.183331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5074cf3e6248668995f266845c8969', 'request_id': 'req-d72fa229-d786-4ae4-8b37-27da61058e00'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:28.191262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5074cf3e6248668995f266845c8969', 'request_id': 'req-d72fa229-d786-4ae4-8b37-27da61058e00'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:28.209851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] The token cache is being invalidate because project 8b5074cf3e6248668995f266845c8969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:28.210190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:28.211041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d72fa229-d786-4ae4-8b37-27da61058e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:28.213162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1161/2328] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:28 2022] DELETE /identity/v3/projects/8b5074cf3e6248668995f266845c8969 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:28.218116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba2c5-de81-434a-8f38-41c55fdefe83 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.218628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba2c5-de81-434a-8f38-41c55fdefe83 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.218917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba2c5-de81-434a-8f38-41c55fdefe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.242676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-074ba2c5-de81-434a-8f38-41c55fdefe83 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:28.243914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-074ba2c5-de81-434a-8f38-41c55fdefe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:28.308400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1168/2329] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:10:28.317683 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0ccd7-9da4-474b-a8bf-91587ab7efa6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.318520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0ccd7-9da4-474b-a8bf-91587ab7efa6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.318808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0ccd7-9da4-474b-a8bf-91587ab7efa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.339066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-98d0ccd7-9da4-474b-a8bf-91587ab7efa6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:28.340346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-98d0ccd7-9da4-474b-a8bf-91587ab7efa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:28.349413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1162/2330] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:28.353419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ba2c5-de81-434a-8f38-41c55fdefe83 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:28.363106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ba2c5-de81-434a-8f38-41c55fdefe83 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:28.368518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:28.369252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.369501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.369827 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.370538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:10:28.370857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:28.372728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:28.407346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1169/2331] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:10:28.413265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0ccd7-9da4-474b-a8bf-91587ab7efa6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:28.424090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0ccd7-9da4-474b-a8bf-91587ab7efa6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:28.429740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075410f-260d-4621-8928-c20b9562ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:28.430494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075410f-260d-4621-8928-c20b9562ddeb admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.430873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075410f-260d-4621-8928-c20b9562ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.431131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075410f-260d-4621-8928-c20b9562ddeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.432041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075410f-260d-4621-8928-c20b9562ddeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:28.434031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075410f-260d-4621-8928-c20b9562ddeb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:28.460762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1163/2332] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:28 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:10:28.464914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:28.475685 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b030c-24ee-4d44-95cb-6b5e2b558462 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:28.548559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1593d-364e-49e1-a5c4-f86e4a96df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:28.549244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1593d-364e-49e1-a5c4-f86e4a96df30 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.549528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1593d-364e-49e1-a5c4-f86e4a96df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.549808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1593d-364e-49e1-a5c4-f86e4a96df30 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.550514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1593d-364e-49e1-a5c4-f86e4a96df30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:28.552675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1593d-364e-49e1-a5c4-f86e4a96df30 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:28.572790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1170/2333] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:28 2022] POST /identity/v3/users => generated 302 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:28.577224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075410f-260d-4621-8928-c20b9562ddeb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:28.590927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075410f-260d-4621-8928-c20b9562ddeb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:28.595714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73aa8c-3d7b-4806-a17f-fb976c8cc593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:28.596435 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73aa8c-3d7b-4806-a17f-fb976c8cc593 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.596727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73aa8c-3d7b-4806-a17f-fb976c8cc593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.597005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73aa8c-3d7b-4806-a17f-fb976c8cc593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.598948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73aa8c-3d7b-4806-a17f-fb976c8cc593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:28.600611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73aa8c-3d7b-4806-a17f-fb976c8cc593 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:28.613036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1164/2334] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:28.618288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1593d-364e-49e1-a5c4-f86e4a96df30 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:28.630612 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1593d-364e-49e1-a5c4-f86e4a96df30 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:28.635782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94ad34-822e-417d-bfbd-a8e1b67da957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:28.636744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94ad34-822e-417d-bfbd-a8e1b67da957 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.637119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94ad34-822e-417d-bfbd-a8e1b67da957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.637503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94ad34-822e-417d-bfbd-a8e1b67da957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.638672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94ad34-822e-417d-bfbd-a8e1b67da957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:28.640451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94ad34-822e-417d-bfbd-a8e1b67da957 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:28.644044 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:28.657421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1171/2335] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:28.660800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73aa8c-3d7b-4806-a17f-fb976c8cc593 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:28.679263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73aa8c-3d7b-4806-a17f-fb976c8cc593 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:28.684282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd33d-8ada-4cc7-8cbd-e802e84ee376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:28.685194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd33d-8ada-4cc7-8cbd-e802e84ee376 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.685478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd33d-8ada-4cc7-8cbd-e802e84ee376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.685760 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fd33d-8ada-4cc7-8cbd-e802e84ee376 admin admin] PATH_INFO: `/v3/projects/c951ed8b274d4e52bc96c67864d7bfbe/users/3cf8740dde274b47a639c9d328df23ce/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.708823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fd33d-8ada-4cc7-8cbd-e802e84ee376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c951ed8b274d4e52bc96c67864d7bfbe, user_id=3cf8740dde274b47a639c9d328df23ce, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:28.711743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fd33d-8ada-4cc7-8cbd-e802e84ee376 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:28.740623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1165/2336] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:28 2022] PUT /identity/v3/projects/c951ed8b274d4e52bc96c67864d7bfbe/users/3cf8740dde274b47a639c9d328df23ce/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:28.751688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdadb1a-7777-4169-b73c-9ae436fd043d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.751939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdadb1a-7777-4169-b73c-9ae436fd043d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.752257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdadb1a-7777-4169-b73c-9ae436fd043d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.784312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3fdadb1a-7777-4169-b73c-9ae436fd043d None None] MFA Rules not processed for user `3cf8740dde274b47a639c9d328df23ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:28.786183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3fdadb1a-7777-4169-b73c-9ae436fd043d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:28.873596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1172/2337] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:28.881784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd33d-8ada-4cc7-8cbd-e802e84ee376 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:28.958593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fd33d-8ada-4cc7-8cbd-e802e84ee376 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:28.963494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad7039-b8c8-41b1-bfd4-ac2e3e5b2540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:28.964084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad7039-b8c8-41b1-bfd4-ac2e3e5b2540 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:28.964395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad7039-b8c8-41b1-bfd4-ac2e3e5b2540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:28.964673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad7039-b8c8-41b1-bfd4-ac2e3e5b2540 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:28.965367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad7039-b8c8-41b1-bfd4-ac2e3e5b2540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:28.967314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad7039-b8c8-41b1-bfd4-ac2e3e5b2540 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:28.993067 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1166/2338] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:28 2022] POST /identity/v3/projects => generated 405 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:28.997318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdadb1a-7777-4169-b73c-9ae436fd043d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:29.009007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdadb1a-7777-4169-b73c-9ae436fd043d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:29.076055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67505bf-d525-43d6-af04-e6ef989660c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:29.076790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67505bf-d525-43d6-af04-e6ef989660c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.077245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67505bf-d525-43d6-af04-e6ef989660c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.077532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67505bf-d525-43d6-af04-e6ef989660c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:29.078239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67505bf-d525-43d6-af04-e6ef989660c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:29.080247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67505bf-d525-43d6-af04-e6ef989660c9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:29.104900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1173/2339] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:28 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:29.106317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad7039-b8c8-41b1-bfd4-ac2e3e5b2540 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:29.118036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad7039-b8c8-41b1-bfd4-ac2e3e5b2540 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:29.122969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde535cd-5f96-44ad-ab5e-d7d4c1c3fc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:29.123875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde535cd-5f96-44ad-ab5e-d7d4c1c3fc63 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.124107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde535cd-5f96-44ad-ab5e-d7d4c1c3fc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.124453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde535cd-5f96-44ad-ab5e-d7d4c1c3fc63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:29.125290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde535cd-5f96-44ad-ab5e-d7d4c1c3fc63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:29.127478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde535cd-5f96-44ad-ab5e-d7d4c1c3fc63 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:29.140537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1167/2340] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:29.145627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67505bf-d525-43d6-af04-e6ef989660c9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:29.155804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67505bf-d525-43d6-af04-e6ef989660c9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:29.160720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4fa74-2876-4f38-a812-2ba998ed4b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:29.161613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4fa74-2876-4f38-a812-2ba998ed4b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.161900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4fa74-2876-4f38-a812-2ba998ed4b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.162176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4fa74-2876-4f38-a812-2ba998ed4b96 admin admin] PATH_INFO: `/v3/projects/b0ec20e7b89d47bcb2ad07e7124f678f/users/9bf0a208bc294df482d77732ff7bb4a7/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:29.177375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4fa74-2876-4f38-a812-2ba998ed4b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ec20e7b89d47bcb2ad07e7124f678f, user_id=9bf0a208bc294df482d77732ff7bb4a7, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:29.180050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4fa74-2876-4f38-a812-2ba998ed4b96 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:29.193604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1174/2341] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:29 2022] PUT /identity/v3/projects/b0ec20e7b89d47bcb2ad07e7124f678f/users/9bf0a208bc294df482d77732ff7bb4a7/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:29.197600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde535cd-5f96-44ad-ab5e-d7d4c1c3fc63 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:29.207681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde535cd-5f96-44ad-ab5e-d7d4c1c3fc63 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:29.226304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:29.280488 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e3454-7de1-43a5-b9eb-0d745627a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:29.281279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3454-7de1-43a5-b9eb-0d745627a672 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.281566 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3454-7de1-43a5-b9eb-0d745627a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.281849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3454-7de1-43a5-b9eb-0d745627a672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:29.282587 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e3454-7de1-43a5-b9eb-0d745627a672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:29.284258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e3454-7de1-43a5-b9eb-0d745627a672 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:29.295226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1168/2342] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:29.299531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4fa74-2876-4f38-a812-2ba998ed4b96 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:29.310727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4fa74-2876-4f38-a812-2ba998ed4b96 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:29.315515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:29.317140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.317631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.317918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] PATH_INFO: `/v3/projects/b0ec20e7b89d47bcb2ad07e7124f678f/users/9bf0a208bc294df482d77732ff7bb4a7/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:29.319892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ec20e7b89d47bcb2ad07e7124f678f, user_id=9bf0a208bc294df482d77732ff7bb4a7, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:29.322639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:29.333059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1175/2343] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:29 2022] PUT /identity/v3/projects/b0ec20e7b89d47bcb2ad07e7124f678f/users/9bf0a208bc294df482d77732ff7bb4a7/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:29.343601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87e319-9daa-46b7-87d2-82492c0efa42 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.343928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87e319-9daa-46b7-87d2-82492c0efa42 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.344260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87e319-9daa-46b7-87d2-82492c0efa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:29.376518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0f87e319-9daa-46b7-87d2-82492c0efa42 None None] MFA Rules not processed for user `9bf0a208bc294df482d77732ff7bb4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:29.378273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0f87e319-9daa-46b7-87d2-82492c0efa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:29.455265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1169/2344] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:29.466541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:29.468535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:29.520931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:29.537648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b04af-ab0f-4afe-b0a5-e44fc0076448 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:29.541345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc856a02-bb07-4ee1-8fa1-7e47a80e273a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:29.541998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc856a02-bb07-4ee1-8fa1-7e47a80e273a service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.542292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc856a02-bb07-4ee1-8fa1-7e47a80e273a service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.542575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc856a02-bb07-4ee1-8fa1-7e47a80e273a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:29.552120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc856a02-bb07-4ee1-8fa1-7e47a80e273a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:29.553597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc856a02-bb07-4ee1-8fa1-7e47a80e273a service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:29.563852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1176/2345] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:29 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:29.713445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87e319-9daa-46b7-87d2-82492c0efa42 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:29.788056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87e319-9daa-46b7-87d2-82492c0efa42 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:29.796321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:29.797558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.797926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.798201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] PATH_INFO: `/v3/users/3cf8740dde274b47a639c9d328df23ce` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:29.799675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf8740dde274b47a639c9d328df23ce)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:29.802333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:29.887210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:29.889567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf8740dde274b47a639c9d328df23ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:29.919981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf8740dde274b47a639c9d328df23ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:29.920892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf8740dde274b47a639c9d328df23ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:29.936990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf8740dde274b47a639c9d328df23ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:29.944128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf8740dde274b47a639c9d328df23ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:29.974482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1170/2346] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:29 2022] DELETE /identity/v3/users/3cf8740dde274b47a639c9d328df23ce => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:29.979203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9506a-dda6-4e69-9f83-b1a301ae5518 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:29.979683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9506a-dda6-4e69-9f83-b1a301ae5518 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:29.979793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9506a-dda6-4e69-9f83-b1a301ae5518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:30.016628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f3c9506a-dda6-4e69-9f83-b1a301ae5518 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:30.016628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f3c9506a-dda6-4e69-9f83-b1a301ae5518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:30.158083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1177/2347] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:30.165914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:30.167704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:30.250512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef8b55-3151-44e5-9a12-8eeec2ffebff admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:30.260641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dcb64-a701-4e2a-9f06-202118addb8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:30.261355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dcb64-a701-4e2a-9f06-202118addb8e service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:30.261760 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dcb64-a701-4e2a-9f06-202118addb8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:30.261976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dcb64-a701-4e2a-9f06-202118addb8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:30.282608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dcb64-a701-4e2a-9f06-202118addb8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:30.284051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dcb64-a701-4e2a-9f06-202118addb8e service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:30.297235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1171/2348] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:30.635880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9506a-dda6-4e69-9f83-b1a301ae5518 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:30.637696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f3c9506a-dda6-4e69-9f83-b1a301ae5518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:30.667531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9506a-dda6-4e69-9f83-b1a301ae5518 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:30.680148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:30.681153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:30.681509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:30.681851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] PATH_INFO: `/v3/projects/c951ed8b274d4e52bc96c67864d7bfbe` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:30.683194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c951ed8b274d4e52bc96c67864d7bfbe)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:30.685246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:30.745342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c951ed8b274d4e52bc96c67864d7bfbe', 'request_id': 'req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.756380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c951ed8b274d4e52bc96c67864d7bfbe', 'request_id': 'req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.773223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c951ed8b274d4e52bc96c67864d7bfbe', 'request_id': 'req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.783750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] The token cache is being invalidate because project c951ed8b274d4e52bc96c67864d7bfbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:30.784074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.784822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.787166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1178/2349] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:30 2022] DELETE /identity/v3/projects/c951ed8b274d4e52bc96c67864d7bfbe => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:30.791322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dcb64-a701-4e2a-9f06-202118addb8e service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:30.802512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dcb64-a701-4e2a-9f06-202118addb8e service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:30.872705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:30.873981 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:30.874434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:30.874872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] PATH_INFO: `/v3/users/9bf0a208bc294df482d77732ff7bb4a7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:30.876366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf0a208bc294df482d77732ff7bb4a7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:30.878638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:30.930520 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf0a208bc294df482d77732ff7bb4a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.946028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf0a208bc294df482d77732ff7bb4a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.947085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf0a208bc294df482d77732ff7bb4a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.950386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:30.960736 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf0a208bc294df482d77732ff7bb4a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.961558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-60dd89d3-4872-480c-a756-7334cb626266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf0a208bc294df482d77732ff7bb4a7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:30.982054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1172/2350] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:30 2022] DELETE /identity/v3/users/9bf0a208bc294df482d77732ff7bb4a7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:31.404850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:31.409710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:31.435967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d2ca9-4efc-45bf-9802-f5ca87d8ec86 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:31.611280 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:31.612971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:31.613888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:31.614665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] PATH_INFO: `/v3/projects/b0ec20e7b89d47bcb2ad07e7124f678f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:31.619574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ec20e7b89d47bcb2ad07e7124f678f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:31.621913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:31.632859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:31.705106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ec20e7b89d47bcb2ad07e7124f678f', 'request_id': 'req-45c83159-22b5-4be5-89a5-3115aeb14615'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:31.723993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ec20e7b89d47bcb2ad07e7124f678f', 'request_id': 'req-45c83159-22b5-4be5-89a5-3115aeb14615'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:31.734652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ec20e7b89d47bcb2ad07e7124f678f', 'request_id': 'req-45c83159-22b5-4be5-89a5-3115aeb14615'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:31.754029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] The token cache is being invalidate because project b0ec20e7b89d47bcb2ad07e7124f678f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:31.754674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:31.755923 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45c83159-22b5-4be5-89a5-3115aeb14615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:31.757821 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1179/2351] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:31 2022] DELETE /identity/v3/projects/b0ec20e7b89d47bcb2ad07e7124f678f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:31.764709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db76b5d-0f91-4db8-be63-622f1cd9bffd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:31.765136 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db76b5d-0f91-4db8-be63-622f1cd9bffd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:31.765460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db76b5d-0f91-4db8-be63-622f1cd9bffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:31.797139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4db76b5d-0f91-4db8-be63-622f1cd9bffd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:31.799590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4db76b5d-0f91-4db8-be63-622f1cd9bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:31.921239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1173/2352] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:31.931064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8374640-9abd-4aee-97d9-c8bb17a41a3c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:31.931363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8374640-9abd-4aee-97d9-c8bb17a41a3c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:31.931651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8374640-9abd-4aee-97d9-c8bb17a41a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:31.950395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f8374640-9abd-4aee-97d9-c8bb17a41a3c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:31.951611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f8374640-9abd-4aee-97d9-c8bb17a41a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:31.960858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1180/2353] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:31.965380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db76b5d-0f91-4db8-be63-622f1cd9bffd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:31.979180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db76b5d-0f91-4db8-be63-622f1cd9bffd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:31.984780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:31.985625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:31.985986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:31.986251 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:31.987272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:10:31.987620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:31.989841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.004541 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1174/2354] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:10:32.009270 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8374640-9abd-4aee-97d9-c8bb17a41a3c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.021645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8374640-9abd-4aee-97d9-c8bb17a41a3c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.029000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43d2bf-b1ba-4ce5-823b-9d5367711ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.029735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43d2bf-b1ba-4ce5-823b-9d5367711ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.030030 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43d2bf-b1ba-4ce5-823b-9d5367711ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.030356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43d2bf-b1ba-4ce5-823b-9d5367711ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.031102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43d2bf-b1ba-4ce5-823b-9d5367711ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.033120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43d2bf-b1ba-4ce5-823b-9d5367711ab1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.074849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1181/2355] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:32 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:32.076436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.090093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd5ce8-0a60-4ee5-a5d0-84942afd6575 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.177919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8132d8c-f086-45ff-ab1a-31f1a8ffb1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.178615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8132d8c-f086-45ff-ab1a-31f1a8ffb1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.178991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8132d8c-f086-45ff-ab1a-31f1a8ffb1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.179184 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8132d8c-f086-45ff-ab1a-31f1a8ffb1f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.180036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8132d8c-f086-45ff-ab1a-31f1a8ffb1f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.182047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8132d8c-f086-45ff-ab1a-31f1a8ffb1f1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.207838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1175/2356] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:32 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:10:32.211966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43d2bf-b1ba-4ce5-823b-9d5367711ab1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.225169 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43d2bf-b1ba-4ce5-823b-9d5367711ab1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.230520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b43ac-2680-4193-9927-88176cb2b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.231190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b43ac-2680-4193-9927-88176cb2b2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.231476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b43ac-2680-4193-9927-88176cb2b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.231754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b43ac-2680-4193-9927-88176cb2b2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.232525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b43ac-2680-4193-9927-88176cb2b2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.234300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b43ac-2680-4193-9927-88176cb2b2a3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.246245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1182/2357] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:32.250356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8132d8c-f086-45ff-ab1a-31f1a8ffb1f1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.261355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8132d8c-f086-45ff-ab1a-31f1a8ffb1f1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.266020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bd100-40fa-411c-b67a-ebacd9ccedca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.266677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bd100-40fa-411c-b67a-ebacd9ccedca admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.266961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bd100-40fa-411c-b67a-ebacd9ccedca admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.267238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bd100-40fa-411c-b67a-ebacd9ccedca admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.267995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bd100-40fa-411c-b67a-ebacd9ccedca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.270532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bd100-40fa-411c-b67a-ebacd9ccedca admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.280480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1176/2358] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:32.284746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b43ac-2680-4193-9927-88176cb2b2a3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.295042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b43ac-2680-4193-9927-88176cb2b2a3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.306543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21925058-3848-44cf-92f9-a6fcb1aaabeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.306543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21925058-3848-44cf-92f9-a6fcb1aaabeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.306543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21925058-3848-44cf-92f9-a6fcb1aaabeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.306543 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21925058-3848-44cf-92f9-a6fcb1aaabeb admin admin] PATH_INFO: `/v3/projects/71ad67c3b7484452a9d99a7a25b6932e/users/a236806f903e4f018d47daf17e1b3065/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.322997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21925058-3848-44cf-92f9-a6fcb1aaabeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ad67c3b7484452a9d99a7a25b6932e, user_id=a236806f903e4f018d47daf17e1b3065, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.325894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21925058-3848-44cf-92f9-a6fcb1aaabeb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.340967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1183/2359] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:32 2022] PUT /identity/v3/projects/71ad67c3b7484452a9d99a7a25b6932e/users/a236806f903e4f018d47daf17e1b3065/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:32.352043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49c3a0-231a-4d1d-8322-29c7a039d60e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.352392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49c3a0-231a-4d1d-8322-29c7a039d60e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.352678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49c3a0-231a-4d1d-8322-29c7a039d60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.383517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dd49c3a0-231a-4d1d-8322-29c7a039d60e None None] MFA Rules not processed for user `a236806f903e4f018d47daf17e1b3065`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:32.384026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dd49c3a0-231a-4d1d-8322-29c7a039d60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:32.430220 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:32.477998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1177/2360] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:32.482973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21925058-3848-44cf-92f9-a6fcb1aaabeb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.594848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21925058-3848-44cf-92f9-a6fcb1aaabeb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.600401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb631ab7-99d1-44d7-a99f-7af485ce2e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.601108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb631ab7-99d1-44d7-a99f-7af485ce2e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.601399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb631ab7-99d1-44d7-a99f-7af485ce2e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.601682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb631ab7-99d1-44d7-a99f-7af485ce2e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.602378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb631ab7-99d1-44d7-a99f-7af485ce2e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.604450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb631ab7-99d1-44d7-a99f-7af485ce2e49 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.644039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1184/2361] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:32 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:10:32.644917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49c3a0-231a-4d1d-8322-29c7a039d60e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.659816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49c3a0-231a-4d1d-8322-29c7a039d60e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.723382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09af0d6-a4e5-4260-a91e-85ad208f97ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.724073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09af0d6-a4e5-4260-a91e-85ad208f97ff admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.724418 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09af0d6-a4e5-4260-a91e-85ad208f97ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.724701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09af0d6-a4e5-4260-a91e-85ad208f97ff admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.725426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09af0d6-a4e5-4260-a91e-85ad208f97ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.727406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09af0d6-a4e5-4260-a91e-85ad208f97ff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.749933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1178/2362] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:32 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:32.754095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb631ab7-99d1-44d7-a99f-7af485ce2e49 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.765403 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb631ab7-99d1-44d7-a99f-7af485ce2e49 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.771261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b34769-27f1-491f-9b30-579aaf013d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.771946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b34769-27f1-491f-9b30-579aaf013d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.772263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b34769-27f1-491f-9b30-579aaf013d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.772548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b34769-27f1-491f-9b30-579aaf013d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.773290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b34769-27f1-491f-9b30-579aaf013d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.774949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b34769-27f1-491f-9b30-579aaf013d72 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.787748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1185/2363] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:32.791740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09af0d6-a4e5-4260-a91e-85ad208f97ff admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.803013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09af0d6-a4e5-4260-a91e-85ad208f97ff admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.807847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b1cd-d10f-48e3-8721-397906c2d17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.808784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b1cd-d10f-48e3-8721-397906c2d17c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.809071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b1cd-d10f-48e3-8721-397906c2d17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.809353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799b1cd-d10f-48e3-8721-397906c2d17c admin admin] PATH_INFO: `/v3/projects/ef6a0672e3464b00a93773ef02d82cab/users/9041e2f5cdf247b6a4e6939fd97a0946/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.824449 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799b1cd-d10f-48e3-8721-397906c2d17c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6a0672e3464b00a93773ef02d82cab, user_id=9041e2f5cdf247b6a4e6939fd97a0946, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.827144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799b1cd-d10f-48e3-8721-397906c2d17c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:32.841053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1179/2364] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:32 2022] PUT /identity/v3/projects/ef6a0672e3464b00a93773ef02d82cab/users/9041e2f5cdf247b6a4e6939fd97a0946/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:32.845232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b34769-27f1-491f-9b30-579aaf013d72 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:32.864775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b34769-27f1-491f-9b30-579aaf013d72 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:32.869314 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:32.976445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f5502-56fd-4dc8-b652-7c596ef3c528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:32.977138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f5502-56fd-4dc8-b652-7c596ef3c528 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:32.977551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f5502-56fd-4dc8-b652-7c596ef3c528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:32.977949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f5502-56fd-4dc8-b652-7c596ef3c528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:32.978795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f5502-56fd-4dc8-b652-7c596ef3c528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:32.980535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f5502-56fd-4dc8-b652-7c596ef3c528 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:33.005449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1186/2365] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:33.006074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b1cd-d10f-48e3-8721-397906c2d17c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:33.017726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799b1cd-d10f-48e3-8721-397906c2d17c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:33.023871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:33.025034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:33.025425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:33.025829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] PATH_INFO: `/v3/projects/ef6a0672e3464b00a93773ef02d82cab/users/9041e2f5cdf247b6a4e6939fd97a0946/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:33.028391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6a0672e3464b00a93773ef02d82cab, user_id=9041e2f5cdf247b6a4e6939fd97a0946, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:33.031179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:33.043966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1180/2366] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:33 2022] PUT /identity/v3/projects/ef6a0672e3464b00a93773ef02d82cab/users/9041e2f5cdf247b6a4e6939fd97a0946/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:10:33.055784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa9bf-ccc2-4f28-b99c-ca1e83a032ea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:33.056205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa9bf-ccc2-4f28-b99c-ca1e83a032ea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:33.056598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa9bf-ccc2-4f28-b99c-ca1e83a032ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:33.096181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ba8fa9bf-ccc2-4f28-b99c-ca1e83a032ea None None] MFA Rules not processed for user `9041e2f5cdf247b6a4e6939fd97a0946`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:33.098776 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:33.100837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:33.102203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ba8fa9bf-ccc2-4f28-b99c-ca1e83a032ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:33.196938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928ca4-59f0-4b2e-9106-9c15e0e34845 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:33.201121 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1187/2367] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:33.203003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:33.203929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:33.204250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:33.204532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] PATH_INFO: `/v3/users/c3bf1694d0284d3d900104304c8f07fd` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:33.205819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3bf1694d0284d3d900104304c8f07fd)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:33.208403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:33.215453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fa9bf-ccc2-4f28-b99c-ca1e83a032ea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:33.217342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ba8fa9bf-ccc2-4f28-b99c-ca1e83a032ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:33.277360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:33.301623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3bf1694d0284d3d900104304c8f07fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:33.311804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3bf1694d0284d3d900104304c8f07fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:33.311982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fa9bf-ccc2-4f28-b99c-ca1e83a032ea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:33.312567 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3bf1694d0284d3d900104304c8f07fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:33.315702 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700dd63-09e5-4331-9957-ed10a62cd423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:33.317176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700dd63-09e5-4331-9957-ed10a62cd423 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:33.317422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700dd63-09e5-4331-9957-ed10a62cd423 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:33.317649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700dd63-09e5-4331-9957-ed10a62cd423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:33.323027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3bf1694d0284d3d900104304c8f07fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:33.324091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3bf1694d0284d3d900104304c8f07fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:33.329897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700dd63-09e5-4331-9957-ed10a62cd423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:33.330380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700dd63-09e5-4331-9957-ed10a62cd423 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:33.342929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1188/2368] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:10:33.352405 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1181/2369] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:33 2022] DELETE /identity/v3/users/c3bf1694d0284d3d900104304c8f07fd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:33.357020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952de058-0f6b-47b8-9343-a541fc9a6d23 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:33.357742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952de058-0f6b-47b8-9343-a541fc9a6d23 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:33.358353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952de058-0f6b-47b8-9343-a541fc9a6d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:33.382266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-952de058-0f6b-47b8-9343-a541fc9a6d23 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:33.383909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-952de058-0f6b-47b8-9343-a541fc9a6d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:33.399897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:33.403133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:33.418222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:33.467790 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1189/2370] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:10:33.479253 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952de058-0f6b-47b8-9343-a541fc9a6d23 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:33.480836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-952de058-0f6b-47b8-9343-a541fc9a6d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:33.487156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039039-d644-43c2-b98f-a9e02edbed8f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:33.491592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:33.492299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:33.492597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:33.492880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:33.494842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:33.574097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952de058-0f6b-47b8-9343-a541fc9a6d23 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:33.579307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:33.579501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc9817-2b82-4931-a268-481ee6988cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:33.580224 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc9817-2b82-4931-a268-481ee6988cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:33.580613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc9817-2b82-4931-a268-481ee6988cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:33.580917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:33.581097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc9817-2b82-4931-a268-481ee6988cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:33.593034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc9817-2b82-4931-a268-481ee6988cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:33.600072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc9817-2b82-4931-a268-481ee6988cb7 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:33.604354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1182/2371] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:33.606698 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1190/2372] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:34.038771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:34.041018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:34.080053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de4df1-473d-48f0-a1f9-7371865a1c60 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:34.131958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:34.132895 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:34.133181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:34.133477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] PATH_INFO: `/v3/projects/7b75d7c218d44df58d1d0f41ea0bc10b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:34.140141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b75d7c218d44df58d1d0f41ea0bc10b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:34.142108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:34.234103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b75d7c218d44df58d1d0f41ea0bc10b', 'request_id': 'req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.244871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b75d7c218d44df58d1d0f41ea0bc10b', 'request_id': 'req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.250717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:34.252763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b75d7c218d44df58d1d0f41ea0bc10b', 'request_id': 'req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.274916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] The token cache is being invalidate because project 7b75d7c218d44df58d1d0f41ea0bc10b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:34.275237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.276558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.278639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1183/2373] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:34 2022] DELETE /identity/v3/projects/7b75d7c218d44df58d1d0f41ea0bc10b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:34.282876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc9817-2b82-4931-a268-481ee6988cb7 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:34.302142 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc9817-2b82-4931-a268-481ee6988cb7 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:34.415000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:34.415876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:34.416196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:34.416474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] PATH_INFO: `/v3/users/2545ccb29e7b45d0b66f3c2558113f6d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:34.417718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=2545ccb29e7b45d0b66f3c2558113f6d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:34.419771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:34.484792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2545ccb29e7b45d0b66f3c2558113f6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.500701 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2545ccb29e7b45d0b66f3c2558113f6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.501769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2545ccb29e7b45d0b66f3c2558113f6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.502433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2545ccb29e7b45d0b66f3c2558113f6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.514996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6464966e-38de-410b-b915-0490aa3f2cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2545ccb29e7b45d0b66f3c2558113f6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:34.517693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:34.530959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1191/2374] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:34 2022] DELETE /identity/v3/users/2545ccb29e7b45d0b66f3c2558113f6d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:34.995171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:34.997023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:35.029864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9bbf4-e030-4ed8-aad1-5f0734f017d4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:35.140232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:35.141341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.141716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.142141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] PATH_INFO: `/v3/projects/9ebd2be4cfc3429c9a49efae8cdabedd` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.143833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ebd2be4cfc3429c9a49efae8cdabedd)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:35.147094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:35.224414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ebd2be4cfc3429c9a49efae8cdabedd', 'request_id': 'req-09d62598-6042-4690-a8cf-8a25a7389c06'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:35.239180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ebd2be4cfc3429c9a49efae8cdabedd', 'request_id': 'req-09d62598-6042-4690-a8cf-8a25a7389c06'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:35.246148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ebd2be4cfc3429c9a49efae8cdabedd', 'request_id': 'req-09d62598-6042-4690-a8cf-8a25a7389c06'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:35.262555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] The token cache is being invalidate because project 9ebd2be4cfc3429c9a49efae8cdabedd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:35.263221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:35.264045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09d62598-6042-4690-a8cf-8a25a7389c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:35.266731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1184/2375] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:34 2022] DELETE /identity/v3/projects/9ebd2be4cfc3429c9a49efae8cdabedd => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:35.272516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74ce67-9bd3-487d-b945-f3da62e33c94 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.273413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74ce67-9bd3-487d-b945-f3da62e33c94 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.273887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74ce67-9bd3-487d-b945-f3da62e33c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.306082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2a74ce67-9bd3-487d-b945-f3da62e33c94 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:35.309794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2a74ce67-9bd3-487d-b945-f3da62e33c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:35.426049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1192/2376] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:35.428303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718294da-e9de-48d2-b782-280abd0bb8f4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.428676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718294da-e9de-48d2-b782-280abd0bb8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.428962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718294da-e9de-48d2-b782-280abd0bb8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.464545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-718294da-e9de-48d2-b782-280abd0bb8f4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:35.465975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-718294da-e9de-48d2-b782-280abd0bb8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:35.480514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1185/2377] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:10:35.485260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74ce67-9bd3-487d-b945-f3da62e33c94 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:35.507171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74ce67-9bd3-487d-b945-f3da62e33c94 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:35.512586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:35.513558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.513558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.513812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.514729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:10:35.515201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:35.517288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:35.538713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1193/2378] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:10:35.540827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718294da-e9de-48d2-b782-280abd0bb8f4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:35.558680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718294da-e9de-48d2-b782-280abd0bb8f4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:35.580354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc75af-5a16-4af5-9f5c-f7d52be8233a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:35.585086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc75af-5a16-4af5-9f5c-f7d52be8233a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.585456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc75af-5a16-4af5-9f5c-f7d52be8233a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.585747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc75af-5a16-4af5-9f5c-f7d52be8233a admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.586727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc75af-5a16-4af5-9f5c-f7d52be8233a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:35.595409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc75af-5a16-4af5-9f5c-f7d52be8233a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:35.631061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1186/2379] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:35 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:35.637313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:35.652377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadf94-6ae1-4cde-8c40-50fc5077f61e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:35.769958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5450f-1e8e-4818-865d-ef24df848c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:35.770706 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5450f-1e8e-4818-865d-ef24df848c11 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.771431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5450f-1e8e-4818-865d-ef24df848c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.771723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5450f-1e8e-4818-865d-ef24df848c11 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.772478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5450f-1e8e-4818-865d-ef24df848c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:35.774533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5450f-1e8e-4818-865d-ef24df848c11 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:35.814878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1194/2380] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:35 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:35.819088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc75af-5a16-4af5-9f5c-f7d52be8233a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:35.832697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc75af-5a16-4af5-9f5c-f7d52be8233a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:35.838376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691b27d-bd35-4ac7-9571-1c7d120686e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:35.839555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691b27d-bd35-4ac7-9571-1c7d120686e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.839964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691b27d-bd35-4ac7-9571-1c7d120686e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.840259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691b27d-bd35-4ac7-9571-1c7d120686e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.841474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691b27d-bd35-4ac7-9571-1c7d120686e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:35.844589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691b27d-bd35-4ac7-9571-1c7d120686e6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:35.858419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1187/2381] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:35.862587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5450f-1e8e-4818-865d-ef24df848c11 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:35.883316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5450f-1e8e-4818-865d-ef24df848c11 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:35.888930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f10a3-4663-4694-9cff-30237ebbb4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:35.889730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f10a3-4663-4694-9cff-30237ebbb4da admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.890260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f10a3-4663-4694-9cff-30237ebbb4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.890617 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f10a3-4663-4694-9cff-30237ebbb4da admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.891448 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f10a3-4663-4694-9cff-30237ebbb4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:35.893615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f10a3-4663-4694-9cff-30237ebbb4da admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:35.907120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1195/2382] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:35.912609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691b27d-bd35-4ac7-9571-1c7d120686e6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:35.923233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691b27d-bd35-4ac7-9571-1c7d120686e6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:35.935459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a74ac-c4b7-44ae-b72d-304d32282fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:35.935459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a74ac-c4b7-44ae-b72d-304d32282fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.935459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a74ac-c4b7-44ae-b72d-304d32282fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.935459 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a74ac-c4b7-44ae-b72d-304d32282fd8 admin admin] PATH_INFO: `/v3/projects/ef13428057b34e2b9b114c67d044fa92/users/af681a35d1d14c3bbbcd94fa38a7bb51/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:35.953622 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a74ac-c4b7-44ae-b72d-304d32282fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef13428057b34e2b9b114c67d044fa92, user_id=af681a35d1d14c3bbbcd94fa38a7bb51, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:35.957713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a74ac-c4b7-44ae-b72d-304d32282fd8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:35.975940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1188/2383] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:35 2022] PUT /identity/v3/projects/ef13428057b34e2b9b114c67d044fa92/users/af681a35d1d14c3bbbcd94fa38a7bb51/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:35.986336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32cc24-e430-4cb9-93fa-53586a3e970c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:35.986747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32cc24-e430-4cb9-93fa-53586a3e970c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:35.986911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32cc24-e430-4cb9-93fa-53586a3e970c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.017641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1a32cc24-e430-4cb9-93fa-53586a3e970c None None] MFA Rules not processed for user `af681a35d1d14c3bbbcd94fa38a7bb51`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:36.019207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1a32cc24-e430-4cb9-93fa-53586a3e970c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:36.074545 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:36.106107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1196/2384] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:36.110228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a74ac-c4b7-44ae-b72d-304d32282fd8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:36.132186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:36.208344 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a74ac-c4b7-44ae-b72d-304d32282fd8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:36.213787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6a7a5-80ee-49b0-b63b-acf5262c34fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:36.214461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a7a5-80ee-49b0-b63b-acf5262c34fc admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.214743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a7a5-80ee-49b0-b63b-acf5262c34fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.215017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6a7a5-80ee-49b0-b63b-acf5262c34fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.215723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6a7a5-80ee-49b0-b63b-acf5262c34fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:36.217755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6a7a5-80ee-49b0-b63b-acf5262c34fc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:36.249283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1189/2385] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:36 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:10:36.255319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32cc24-e430-4cb9-93fa-53586a3e970c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:36.270413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32cc24-e430-4cb9-93fa-53586a3e970c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:36.332056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc9e9d-73be-4ec3-9786-94e77873b637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:36.333051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc9e9d-73be-4ec3-9786-94e77873b637 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.333335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc9e9d-73be-4ec3-9786-94e77873b637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.333620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc9e9d-73be-4ec3-9786-94e77873b637 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.334411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc9e9d-73be-4ec3-9786-94e77873b637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:36.336392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc9e9d-73be-4ec3-9786-94e77873b637 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:36.358503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1197/2386] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:36 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:36.362508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6a7a5-80ee-49b0-b63b-acf5262c34fc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:36.373207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6a7a5-80ee-49b0-b63b-acf5262c34fc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:36.379168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445966d-9c97-4d19-80ca-f474ceb3d96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:36.380141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445966d-9c97-4d19-80ca-f474ceb3d96e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.380458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445966d-9c97-4d19-80ca-f474ceb3d96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.380738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445966d-9c97-4d19-80ca-f474ceb3d96e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.381477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445966d-9c97-4d19-80ca-f474ceb3d96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:36.383069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445966d-9c97-4d19-80ca-f474ceb3d96e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:36.398133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1190/2387] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:36.401508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc9e9d-73be-4ec3-9786-94e77873b637 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:36.413311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc9e9d-73be-4ec3-9786-94e77873b637 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:36.419194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a45568-33ef-4b5d-b03b-0466a93621f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:36.420016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a45568-33ef-4b5d-b03b-0466a93621f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.420345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a45568-33ef-4b5d-b03b-0466a93621f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.420626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a45568-33ef-4b5d-b03b-0466a93621f8 admin admin] PATH_INFO: `/v3/projects/460ad6fc1568461d8a8ed1052446fd32/users/8e648cd53b33456ab479375c9ecdbe57/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.439160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a45568-33ef-4b5d-b03b-0466a93621f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460ad6fc1568461d8a8ed1052446fd32, user_id=8e648cd53b33456ab479375c9ecdbe57, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:36.442668 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a45568-33ef-4b5d-b03b-0466a93621f8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:36.459210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1198/2388] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:36 2022] PUT /identity/v3/projects/460ad6fc1568461d8a8ed1052446fd32/users/8e648cd53b33456ab479375c9ecdbe57/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:10:36.463887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445966d-9c97-4d19-80ca-f474ceb3d96e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:36.475414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445966d-9c97-4d19-80ca-f474ceb3d96e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:36.540359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009840dc-bbba-4721-adb4-078faeb7cf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:36.541042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009840dc-bbba-4721-adb4-078faeb7cf7e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.541328 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009840dc-bbba-4721-adb4-078faeb7cf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.541607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009840dc-bbba-4721-adb4-078faeb7cf7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.542340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009840dc-bbba-4721-adb4-078faeb7cf7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:36.544043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009840dc-bbba-4721-adb4-078faeb7cf7e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:36.555184 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1191/2389] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:36.559548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a45568-33ef-4b5d-b03b-0466a93621f8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:36.570412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a45568-33ef-4b5d-b03b-0466a93621f8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:36.575367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:36.576388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.577109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.577109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] PATH_INFO: `/v3/projects/460ad6fc1568461d8a8ed1052446fd32/users/8e648cd53b33456ab479375c9ecdbe57/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.579524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460ad6fc1568461d8a8ed1052446fd32, user_id=8e648cd53b33456ab479375c9ecdbe57, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:36.583220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:36.596829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1199/2390] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:36 2022] PUT /identity/v3/projects/460ad6fc1568461d8a8ed1052446fd32/users/8e648cd53b33456ab479375c9ecdbe57/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:36.608091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338571-3ce9-4832-8942-324229a3057f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.608414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338571-3ce9-4832-8942-324229a3057f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.608705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338571-3ce9-4832-8942-324229a3057f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.635645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b3338571-3ce9-4832-8942-324229a3057f None None] MFA Rules not processed for user `8e648cd53b33456ab479375c9ecdbe57`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:36.636897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b3338571-3ce9-4832-8942-324229a3057f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:36.706690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1192/2391] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:36.718016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:36.719797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:36.781015 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39cc75-aba1-42fb-b468-ce0aaddd1518 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:36.785267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:36.786262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.786638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.786921 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.796645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:36.798074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:36.808911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1200/2392] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:36.885727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3338571-3ce9-4832-8942-324229a3057f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:36.896609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3338571-3ce9-4832-8942-324229a3057f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:36.900503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97dfc3-a8c5-459b-be67-b1dc62f5a819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:36.901144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97dfc3-a8c5-459b-be67-b1dc62f5a819 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:36.901427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97dfc3-a8c5-459b-be67-b1dc62f5a819 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:36.901705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97dfc3-a8c5-459b-be67-b1dc62f5a819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:36.911925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:36.978336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97dfc3-a8c5-459b-be67-b1dc62f5a819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:36.979753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97dfc3-a8c5-459b-be67-b1dc62f5a819 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:36.990854 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1193/2393] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:37.492441 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:37.493865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:37.548103 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:37.571233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af49f9-4fc4-4a33-aa01-e9e642e27e87 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:37.577250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:37.578239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:37.578531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:37.578803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] PATH_INFO: `/v3/users/a236806f903e4f018d47daf17e1b3065` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:37.580672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] RBAC: Authorizing `identity:delete_user(user_id=a236806f903e4f018d47daf17e1b3065)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:37.583839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:37.641539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a236806f903e4f018d47daf17e1b3065'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:37.669279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a236806f903e4f018d47daf17e1b3065'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:37.670275 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a236806f903e4f018d47daf17e1b3065'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:37.670838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a236806f903e4f018d47daf17e1b3065'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:37.684226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a236806f903e4f018d47daf17e1b3065'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:37.719544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1201/2394] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:37 2022] DELETE /identity/v3/users/a236806f903e4f018d47daf17e1b3065 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:37.724312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d4935-f588-4739-9700-4665be3cd5a9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:37.724811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d4935-f588-4739-9700-4665be3cd5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:37.724932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d4935-f588-4739-9700-4665be3cd5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:37.746064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1e9d4935-f588-4739-9700-4665be3cd5a9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:37.747455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1e9d4935-f588-4739-9700-4665be3cd5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:37.811046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1194/2395] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:10:37.821784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:37.823217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:37.881037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac4121-8438-4fdc-b8ab-1661929249df admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:37.884568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959876f-be7f-4055-8138-e24d7043e9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:37.885211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959876f-be7f-4055-8138-e24d7043e9fa service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:37.885553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959876f-be7f-4055-8138-e24d7043e9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:37.885872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959876f-be7f-4055-8138-e24d7043e9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:37.896263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959876f-be7f-4055-8138-e24d7043e9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:37.897797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959876f-be7f-4055-8138-e24d7043e9fa service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:37.907364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1202/2396] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:38.198084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d4935-f588-4739-9700-4665be3cd5a9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:38.200424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1e9d4935-f588-4739-9700-4665be3cd5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:38.226371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d4935-f588-4739-9700-4665be3cd5a9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:38.236090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:38.237089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:38.237592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:38.238608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] PATH_INFO: `/v3/projects/71ad67c3b7484452a9d99a7a25b6932e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:38.240235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ad67c3b7484452a9d99a7a25b6932e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:38.243029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:38.306851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ad67c3b7484452a9d99a7a25b6932e', 'request_id': 'req-c142a46f-c94a-4a74-99f1-d2145321987a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.313841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ad67c3b7484452a9d99a7a25b6932e', 'request_id': 'req-c142a46f-c94a-4a74-99f1-d2145321987a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.320919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ad67c3b7484452a9d99a7a25b6932e', 'request_id': 'req-c142a46f-c94a-4a74-99f1-d2145321987a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.337041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] The token cache is being invalidate because project 71ad67c3b7484452a9d99a7a25b6932e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:38.337335 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.337992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.339556 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1195/2397] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:38 2022] DELETE /identity/v3/projects/71ad67c3b7484452a9d99a7a25b6932e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:38.343720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959876f-be7f-4055-8138-e24d7043e9fa service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:38.354774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959876f-be7f-4055-8138-e24d7043e9fa service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:38.419338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:38.420454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:38.420743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:38.421022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] PATH_INFO: `/v3/users/9041e2f5cdf247b6a4e6939fd97a0946` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:38.422213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9041e2f5cdf247b6a4e6939fd97a0946)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:38.424344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:38.433524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:38.481922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9041e2f5cdf247b6a4e6939fd97a0946'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.496001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9041e2f5cdf247b6a4e6939fd97a0946'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.496775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9041e2f5cdf247b6a4e6939fd97a0946'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.497391 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9041e2f5cdf247b6a4e6939fd97a0946'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.504578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4300c8b5-cba0-4954-b8a2-7f4bb4b4b5e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9041e2f5cdf247b6a4e6939fd97a0946'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.518536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1203/2398] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:38 2022] DELETE /identity/v3/users/9041e2f5cdf247b6a4e6939fd97a0946 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:38.824897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:38.826897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:38.843025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142a46f-c94a-4a74-99f1-d2145321987a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:38.878163 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:38.911071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:38.912305 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:38.912593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:38.912871 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] PATH_INFO: `/v3/projects/ef6a0672e3464b00a93773ef02d82cab` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:38.914084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6a0672e3464b00a93773ef02d82cab)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:38.916068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:38.975697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6a0672e3464b00a93773ef02d82cab', 'request_id': 'req-03cb2aa2-522a-42e1-813a-e0a57fbb0149'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.982374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6a0672e3464b00a93773ef02d82cab', 'request_id': 'req-03cb2aa2-522a-42e1-813a-e0a57fbb0149'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:38.989601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6a0672e3464b00a93773ef02d82cab', 'request_id': 'req-03cb2aa2-522a-42e1-813a-e0a57fbb0149'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:39.004396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] The token cache is being invalidate because project ef6a0672e3464b00a93773ef02d82cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:39.004691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:39.005402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03cb2aa2-522a-42e1-813a-e0a57fbb0149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:39.007016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1196/2399] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:38 2022] DELETE /identity/v3/projects/ef6a0672e3464b00a93773ef02d82cab => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:39.011974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e37f0-f0ef-4616-a22d-4db688df09b1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.012318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e37f0-f0ef-4616-a22d-4db688df09b1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.012582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e37f0-f0ef-4616-a22d-4db688df09b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.031160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9c5e37f0-f0ef-4616-a22d-4db688df09b1 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:39.032397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9c5e37f0-f0ef-4616-a22d-4db688df09b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:39.091605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1204/2400] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:39.099927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5d45-e5f8-4e50-ade5-99481fe8b085 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.100243 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5d45-e5f8-4e50-ade5-99481fe8b085 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.100528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf5d45-e5f8-4e50-ade5-99481fe8b085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.119552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ebaf5d45-e5f8-4e50-ade5-99481fe8b085 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:39.120793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ebaf5d45-e5f8-4e50-ade5-99481fe8b085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:39.128908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1197/2401] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:39.133080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e37f0-f0ef-4616-a22d-4db688df09b1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.142935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e37f0-f0ef-4616-a22d-4db688df09b1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.147961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.148681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.148970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.149251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.149976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:10:39.150289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.151968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.164236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1205/2402] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:10:39.168402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf5d45-e5f8-4e50-ade5-99481fe8b085 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.178602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf5d45-e5f8-4e50-ade5-99481fe8b085 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.183300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5528675-5a5c-433c-82c0-69a3df28e444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.183948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5528675-5a5c-433c-82c0-69a3df28e444 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.184268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5528675-5a5c-433c-82c0-69a3df28e444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.184542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5528675-5a5c-433c-82c0-69a3df28e444 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.185235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5528675-5a5c-433c-82c0-69a3df28e444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.187156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5528675-5a5c-433c-82c0-69a3df28e444 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.215498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1198/2403] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:39 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:39.219759 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.231168 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bf4a4-d584-4173-91f4-cc7721f63c81 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.293017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7a459-8816-45c2-8364-d62c5023c30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.293797 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a459-8816-45c2-8364-d62c5023c30b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.294069 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a459-8816-45c2-8364-d62c5023c30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.294350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a459-8816-45c2-8364-d62c5023c30b admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.295057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7a459-8816-45c2-8364-d62c5023c30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.297078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7a459-8816-45c2-8364-d62c5023c30b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.318206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1206/2404] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:39 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:39.322663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5528675-5a5c-433c-82c0-69a3df28e444 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.333583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5528675-5a5c-433c-82c0-69a3df28e444 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.338304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0c377-50dd-44aa-8bfd-98087f36d17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.338966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c377-50dd-44aa-8bfd-98087f36d17e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.339250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c377-50dd-44aa-8bfd-98087f36d17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.339528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0c377-50dd-44aa-8bfd-98087f36d17e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.340296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0c377-50dd-44aa-8bfd-98087f36d17e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.341886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0c377-50dd-44aa-8bfd-98087f36d17e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.352901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1199/2405] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:39.357003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7a459-8816-45c2-8364-d62c5023c30b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.362174 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:39.368410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7a459-8816-45c2-8364-d62c5023c30b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.374982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9fe9a-91bd-4a9c-b563-fe5938b2f9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.375717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9fe9a-91bd-4a9c-b563-fe5938b2f9db admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.376082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9fe9a-91bd-4a9c-b563-fe5938b2f9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.376394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9fe9a-91bd-4a9c-b563-fe5938b2f9db admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.377162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9fe9a-91bd-4a9c-b563-fe5938b2f9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.378927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9fe9a-91bd-4a9c-b563-fe5938b2f9db admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.389697 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1207/2406] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:39.394899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0c377-50dd-44aa-8bfd-98087f36d17e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.404888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0c377-50dd-44aa-8bfd-98087f36d17e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.410231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518153d8-ce15-413d-988b-e7cb723af3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.411110 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518153d8-ce15-413d-988b-e7cb723af3a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.411397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518153d8-ce15-413d-988b-e7cb723af3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.411676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518153d8-ce15-413d-988b-e7cb723af3a2 admin admin] PATH_INFO: `/v3/projects/32006abfeddd41f5ba0643dd50a3d4e2/users/d53b540119174d7f88ce866e8821e4d8/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.426545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518153d8-ce15-413d-988b-e7cb723af3a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32006abfeddd41f5ba0643dd50a3d4e2, user_id=d53b540119174d7f88ce866e8821e4d8, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.429245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518153d8-ce15-413d-988b-e7cb723af3a2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.441999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1200/2407] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:39 2022] PUT /identity/v3/projects/32006abfeddd41f5ba0643dd50a3d4e2/users/d53b540119174d7f88ce866e8821e4d8/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:39.452463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba1a5c-576f-44ee-a3b5-e2ec70e26519 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.452757 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba1a5c-576f-44ee-a3b5-e2ec70e26519 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.453059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba1a5c-576f-44ee-a3b5-e2ec70e26519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.480007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0aba1a5c-576f-44ee-a3b5-e2ec70e26519 None None] MFA Rules not processed for user `d53b540119174d7f88ce866e8821e4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:39.481312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0aba1a5c-576f-44ee-a3b5-e2ec70e26519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:39.544265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1208/2408] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:39.549008 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518153d8-ce15-413d-988b-e7cb723af3a2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.611579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518153d8-ce15-413d-988b-e7cb723af3a2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.616861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02c867-df89-42a9-9914-d71e88dd4564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.617516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02c867-df89-42a9-9914-d71e88dd4564 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.617802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02c867-df89-42a9-9914-d71e88dd4564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.618082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02c867-df89-42a9-9914-d71e88dd4564 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.618768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02c867-df89-42a9-9914-d71e88dd4564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.620712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02c867-df89-42a9-9914-d71e88dd4564 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.623931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:39.652602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1201/2409] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:39 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:39.653881 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba1a5c-576f-44ee-a3b5-e2ec70e26519 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.668691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba1a5c-576f-44ee-a3b5-e2ec70e26519 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.729703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.730383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.730666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.730949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.731715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.733651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.756041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1209/2410] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:39 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:10:39.759901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02c867-df89-42a9-9914-d71e88dd4564 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.771367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02c867-df89-42a9-9914-d71e88dd4564 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.772277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.775473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:39.779993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22c976-da2c-4715-b549-411a7b4f38aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.780706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c976-da2c-4715-b549-411a7b4f38aa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.781089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c976-da2c-4715-b549-411a7b4f38aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.781265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22c976-da2c-4715-b549-411a7b4f38aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.782005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22c976-da2c-4715-b549-411a7b4f38aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.783972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22c976-da2c-4715-b549-411a7b4f38aa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.789207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce2aef-b950-4148-b22b-2a30265c00f3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.794383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.795244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1202/2411] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:39.796275 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.796555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.796837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] PATH_INFO: `/v3/users/af681a35d1d14c3bbbcd94fa38a7bb51` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.798864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af681a35d1d14c3bbbcd94fa38a7bb51)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.799339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22c976-da2c-4715-b549-411a7b4f38aa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.802423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.810330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22c976-da2c-4715-b549-411a7b4f38aa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.815341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7da69-1c69-45b0-b4ac-7270e6762894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:39.816934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7da69-1c69-45b0-b4ac-7270e6762894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.817294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7da69-1c69-45b0-b4ac-7270e6762894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.817633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7da69-1c69-45b0-b4ac-7270e6762894 admin admin] PATH_INFO: `/v3/projects/ef31b84e0834498c9a3390710e68375c/users/85f4a01fb408423382c312f6a95d360d/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.837027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7da69-1c69-45b0-b4ac-7270e6762894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef31b84e0834498c9a3390710e68375c, user_id=85f4a01fb408423382c312f6a95d360d, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:39.839739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7da69-1c69-45b0-b4ac-7270e6762894 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:39.852855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1203/2412] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:39 2022] PUT /identity/v3/projects/ef31b84e0834498c9a3390710e68375c/users/85f4a01fb408423382c312f6a95d360d/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:10:39.856511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7da69-1c69-45b0-b4ac-7270e6762894 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:39.859705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af681a35d1d14c3bbbcd94fa38a7bb51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:39.873292 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af681a35d1d14c3bbbcd94fa38a7bb51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:39.874154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af681a35d1d14c3bbbcd94fa38a7bb51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:39.875169 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af681a35d1d14c3bbbcd94fa38a7bb51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:39.876608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:39.888084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45406868-c46b-4dbe-ab3a-eb8de8e7b638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af681a35d1d14c3bbbcd94fa38a7bb51'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:39.903024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1210/2413] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:39 2022] DELETE /identity/v3/users/af681a35d1d14c3bbbcd94fa38a7bb51 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:10:39.907293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c44ad0-294f-44f2-86b2-c4000be28375 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:39.907590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c44ad0-294f-44f2-86b2-c4000be28375 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:39.907870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c44ad0-294f-44f2-86b2-c4000be28375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:39.924416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7da69-1c69-45b0-b4ac-7270e6762894 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:39.926672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-68c44ad0-294f-44f2-86b2-c4000be28375 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:39.927808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-68c44ad0-294f-44f2-86b2-c4000be28375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:39.997425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1211/2414] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:10:40.006680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c44ad0-294f-44f2-86b2-c4000be28375 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:40.008231 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-68c44ad0-294f-44f2-86b2-c4000be28375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:40.016311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:40.017077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:40.017432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:40.017789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:40.018786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:40.021018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:40.031487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1204/2415] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:10:40.037298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:40.040338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:40.054038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccad3c3-7ddf-4848-a6d3-7d6162777e24 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:40.061249 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee359e-69d3-4895-b39c-5c9ebf8df99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:40.062395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee359e-69d3-4895-b39c-5c9ebf8df99e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:40.062517 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee359e-69d3-4895-b39c-5c9ebf8df99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:40.062721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee359e-69d3-4895-b39c-5c9ebf8df99e admin admin] PATH_INFO: `/v3/projects/ef31b84e0834498c9a3390710e68375c/users/85f4a01fb408423382c312f6a95d360d/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:40.064894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee359e-69d3-4895-b39c-5c9ebf8df99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef31b84e0834498c9a3390710e68375c, user_id=85f4a01fb408423382c312f6a95d360d, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:40.068677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee359e-69d3-4895-b39c-5c9ebf8df99e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:40.073773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c44ad0-294f-44f2-86b2-c4000be28375 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:40.082235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1205/2416] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:40 2022] PUT /identity/v3/projects/ef31b84e0834498c9a3390710e68375c/users/85f4a01fb408423382c312f6a95d360d/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:40.094248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2aaa6-2999-4d9e-aad0-05cea30c8a57 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:40.094575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2aaa6-2999-4d9e-aad0-05cea30c8a57 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:40.094819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2aaa6-2999-4d9e-aad0-05cea30c8a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:40.131702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-88d2aaa6-2999-4d9e-aad0-05cea30c8a57 None None] MFA Rules not processed for user `85f4a01fb408423382c312f6a95d360d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:40.133324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-88d2aaa6-2999-4d9e-aad0-05cea30c8a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:40.145479 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04873-7192-4597-b316-c00dcebe8329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:40.146223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04873-7192-4597-b316-c00dcebe8329 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:40.146586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04873-7192-4597-b316-c00dcebe8329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:40.146938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a04873-7192-4597-b316-c00dcebe8329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:40.169670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:40.200636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:40.214185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a04873-7192-4597-b316-c00dcebe8329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:40.214428 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1206/2417] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:40.216684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a04873-7192-4597-b316-c00dcebe8329 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:40.226151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2aaa6-2999-4d9e-aad0-05cea30c8a57 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:40.227718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-88d2aaa6-2999-4d9e-aad0-05cea30c8a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:40.228271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1212/2418] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:40.293394 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2aaa6-2999-4d9e-aad0-05cea30c8a57 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:40.297402 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:40.298353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:40.298734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:40.299114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:40.311313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:40.312963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:40.324818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1207/2419] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:10:40.437784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04873-7192-4597-b316-c00dcebe8329 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:40.447993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a04873-7192-4597-b316-c00dcebe8329 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:40.452972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:40.453640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:40.453928 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:40.454208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:40.456460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:40.532938 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:40.534695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:40.536875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:40.539044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:40.548700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1213/2420] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:40.561330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25488897-3a65-48a1-9a83-c5c0da9c336e service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:40.566798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:40.567745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:40.568275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:40.568710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] PATH_INFO: `/v3/projects/ef13428057b34e2b9b114c67d044fa92` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:40.570001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef13428057b34e2b9b114c67d044fa92)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:40.572221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:40.640239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef13428057b34e2b9b114c67d044fa92', 'request_id': 'req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.647275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef13428057b34e2b9b114c67d044fa92', 'request_id': 'req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.653287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef13428057b34e2b9b114c67d044fa92', 'request_id': 'req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.666393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] The token cache is being invalidate because project ef13428057b34e2b9b114c67d044fa92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:40.666681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.667310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.669006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1208/2421] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:40 2022] DELETE /identity/v3/projects/ef13428057b34e2b9b114c67d044fa92 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:40.672977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:40.684858 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9fe3e-c2a4-4616-b8f5-6e64ab13adc9 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:40.751659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:40.753193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:40.753758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:40.754131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] PATH_INFO: `/v3/users/8e648cd53b33456ab479375c9ecdbe57` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:40.755506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e648cd53b33456ab479375c9ecdbe57)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:40.757532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:40.813395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e648cd53b33456ab479375c9ecdbe57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.839738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e648cd53b33456ab479375c9ecdbe57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.839738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e648cd53b33456ab479375c9ecdbe57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.839738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e648cd53b33456ab479375c9ecdbe57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.866217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-97826c23-b193-44c3-8316-752faeb90df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e648cd53b33456ab479375c9ecdbe57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:40.900589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1214/2422] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:40 2022] DELETE /identity/v3/users/8e648cd53b33456ab479375c9ecdbe57 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:41.188153 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:41.189850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:41.205330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e1abf-88a3-449f-af89-306b1b0dc2b5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:41.262838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:41.275573 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:41.276505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.277048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.277415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] PATH_INFO: `/v3/projects/460ad6fc1568461d8a8ed1052446fd32` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.278781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=460ad6fc1568461d8a8ed1052446fd32)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:41.281218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:41.332665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '460ad6fc1568461d8a8ed1052446fd32', 'request_id': 'req-a15840c1-6fe9-4c3d-9022-73d55779fc9b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:41.339001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '460ad6fc1568461d8a8ed1052446fd32', 'request_id': 'req-a15840c1-6fe9-4c3d-9022-73d55779fc9b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:41.344578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '460ad6fc1568461d8a8ed1052446fd32', 'request_id': 'req-a15840c1-6fe9-4c3d-9022-73d55779fc9b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:41.358870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] The token cache is being invalidate because project 460ad6fc1568461d8a8ed1052446fd32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:41.359162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:41.359777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a15840c1-6fe9-4c3d-9022-73d55779fc9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:41.361360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1209/2423] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:41 2022] DELETE /identity/v3/projects/460ad6fc1568461d8a8ed1052446fd32 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:41.366358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2d70-c48a-4f27-ad17-97ddf3072fba None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.366649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2d70-c48a-4f27-ad17-97ddf3072fba None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.366929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2d70-c48a-4f27-ad17-97ddf3072fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.380835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:41.387862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5ccd2d70-c48a-4f27-ad17-97ddf3072fba None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:41.389894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5ccd2d70-c48a-4f27-ad17-97ddf3072fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:41.451213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1215/2424] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:41.459829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfaeb8-e321-482e-87d5-0600f4857270 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.460120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfaeb8-e321-482e-87d5-0600f4857270 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.460452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfaeb8-e321-482e-87d5-0600f4857270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.479390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e5cfaeb8-e321-482e-87d5-0600f4857270 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:41.480633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e5cfaeb8-e321-482e-87d5-0600f4857270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:41.488808 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1210/2425] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:41.493157 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd2d70-c48a-4f27-ad17-97ddf3072fba None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:41.502796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd2d70-c48a-4f27-ad17-97ddf3072fba None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:41.507506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:41.508183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.508476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.508758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.509480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:10:41.509795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:41.511414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:41.523676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1216/2426] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:10:41.527928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfaeb8-e321-482e-87d5-0600f4857270 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:41.537982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfaeb8-e321-482e-87d5-0600f4857270 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:41.542555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca153a-c3ad-49c7-b2a8-7bd2e2b9af69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:41.543234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca153a-c3ad-49c7-b2a8-7bd2e2b9af69 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.543511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca153a-c3ad-49c7-b2a8-7bd2e2b9af69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.543787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca153a-c3ad-49c7-b2a8-7bd2e2b9af69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.544510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca153a-c3ad-49c7-b2a8-7bd2e2b9af69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:41.546408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca153a-c3ad-49c7-b2a8-7bd2e2b9af69 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:41.572056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1211/2427] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:41 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:10:41.576034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:41.587262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b79e63-d0b9-4e0a-b0a3-ba3a70bdbf3d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:41.642124 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ce8ce-43be-4b68-89b8-e58f9a553f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:41.642799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ce8ce-43be-4b68-89b8-e58f9a553f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.643327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ce8ce-43be-4b68-89b8-e58f9a553f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.643601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ce8ce-43be-4b68-89b8-e58f9a553f75 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.645214 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ce8ce-43be-4b68-89b8-e58f9a553f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:41.647125 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ce8ce-43be-4b68-89b8-e58f9a553f75 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:41.666708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1217/2428] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:41 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:41.670967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca153a-c3ad-49c7-b2a8-7bd2e2b9af69 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:41.681572 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca153a-c3ad-49c7-b2a8-7bd2e2b9af69 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:41.686397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bec5c1-4559-4ec8-929f-7ffc4c453dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:41.687098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bec5c1-4559-4ec8-929f-7ffc4c453dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.687381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bec5c1-4559-4ec8-929f-7ffc4c453dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.687690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bec5c1-4559-4ec8-929f-7ffc4c453dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.688476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bec5c1-4559-4ec8-929f-7ffc4c453dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:41.690108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bec5c1-4559-4ec8-929f-7ffc4c453dbf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:41.701527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1212/2429] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:41.705657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ce8ce-43be-4b68-89b8-e58f9a553f75 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:41.716793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ce8ce-43be-4b68-89b8-e58f9a553f75 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:41.721674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f5c7c-6f31-4cd5-9c57-314265e54364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:41.722365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f5c7c-6f31-4cd5-9c57-314265e54364 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.722743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f5c7c-6f31-4cd5-9c57-314265e54364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.723029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f5c7c-6f31-4cd5-9c57-314265e54364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.723788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f5c7c-6f31-4cd5-9c57-314265e54364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:41.725429 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f5c7c-6f31-4cd5-9c57-314265e54364 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:41.735550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1218/2430] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:41.739918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bec5c1-4559-4ec8-929f-7ffc4c453dbf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:41.750044 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bec5c1-4559-4ec8-929f-7ffc4c453dbf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:41.755437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:41.756352 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.756637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.756919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] PATH_INFO: `/v3/projects/5b976277ecde412ab4b17b2d17f14340/users/39a41a7b2ea64ca3ad13e70a83bf28ca/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.772687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b976277ecde412ab4b17b2d17f14340, user_id=39a41a7b2ea64ca3ad13e70a83bf28ca, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:41.775372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:41.787961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1213/2431] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:41 2022] PUT /identity/v3/projects/5b976277ecde412ab4b17b2d17f14340/users/39a41a7b2ea64ca3ad13e70a83bf28ca/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:41.793067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2591-b126-493e-9870-99b3f9903b6c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.793360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2591-b126-493e-9870-99b3f9903b6c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.793641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a2591-b126-493e-9870-99b3f9903b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.811487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-663a2591-b126-493e-9870-99b3f9903b6c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:41.812770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-663a2591-b126-493e-9870-99b3f9903b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:41.872529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1219/2432] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:41.882807 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:41.884596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:41.944753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a338dd-0a08-4a96-a6bf-e873817c6b12 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:41.948578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0726ac-ee55-4a1b-a10e-351ac26c7483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:41.949325 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0726ac-ee55-4a1b-a10e-351ac26c7483 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:41.949617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0726ac-ee55-4a1b-a10e-351ac26c7483 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:41.949899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0726ac-ee55-4a1b-a10e-351ac26c7483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:41.961068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0726ac-ee55-4a1b-a10e-351ac26c7483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:41.962531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0726ac-ee55-4a1b-a10e-351ac26c7483 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:41.973445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1214/2433] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:42.761050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c356c-ee23-4daa-bce4-e3e698b3db63 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:42.761634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c356c-ee23-4daa-bce4-e3e698b3db63 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:42.761634 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c356c-ee23-4daa-bce4-e3e698b3db63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:42.789032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:42.793352 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c44c356c-ee23-4daa-bce4-e3e698b3db63 None None] MFA Rules not processed for user `39a41a7b2ea64ca3ad13e70a83bf28ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:42.794631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c44c356c-ee23-4daa-bce4-e3e698b3db63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:42.872101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1220/2434] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:42.876862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0726ac-ee55-4a1b-a10e-351ac26c7483 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:42.887254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0726ac-ee55-4a1b-a10e-351ac26c7483 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:42.891910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36a68c-0522-4b9a-8638-4523106ecd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:42.892629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36a68c-0522-4b9a-8638-4523106ecd26 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:42.892913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36a68c-0522-4b9a-8638-4523106ecd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:42.893460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36a68c-0522-4b9a-8638-4523106ecd26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:42.895020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36a68c-0522-4b9a-8638-4523106ecd26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:42.897789 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36a68c-0522-4b9a-8638-4523106ecd26 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:42.919433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:42.927874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1215/2435] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:42 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:42.932310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c356c-ee23-4daa-bce4-e3e698b3db63 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:42.943371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c356c-ee23-4daa-bce4-e3e698b3db63 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:43.000699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e452d4-ac10-4958-bb66-414d70cf1def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:43.001381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e452d4-ac10-4958-bb66-414d70cf1def admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.001672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e452d4-ac10-4958-bb66-414d70cf1def admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.001955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e452d4-ac10-4958-bb66-414d70cf1def admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.002660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e452d4-ac10-4958-bb66-414d70cf1def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:43.004704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e452d4-ac10-4958-bb66-414d70cf1def admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:43.029389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1221/2436] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:42 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:10:43.030531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36a68c-0522-4b9a-8638-4523106ecd26 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:43.042486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36a68c-0522-4b9a-8638-4523106ecd26 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:43.047241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f26629-2777-411a-b7b7-29f735c38c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:43.047910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f26629-2777-411a-b7b7-29f735c38c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.048308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f26629-2777-411a-b7b7-29f735c38c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.048639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f26629-2777-411a-b7b7-29f735c38c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.049377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f26629-2777-411a-b7b7-29f735c38c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:43.051002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f26629-2777-411a-b7b7-29f735c38c30 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:43.061812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1216/2437] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:10:43.066829 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e452d4-ac10-4958-bb66-414d70cf1def admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:43.076932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e452d4-ac10-4958-bb66-414d70cf1def admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:43.082311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a4288-5ce9-4e91-b46f-c752c8cc60b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:43.083198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a4288-5ce9-4e91-b46f-c752c8cc60b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.083486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a4288-5ce9-4e91-b46f-c752c8cc60b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.083770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a4288-5ce9-4e91-b46f-c752c8cc60b3 admin admin] PATH_INFO: `/v3/projects/c6ea5cb981164e608a1111402c05fd63/users/dbc44a15b5594b599f61ab1e41244619/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.099778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a4288-5ce9-4e91-b46f-c752c8cc60b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ea5cb981164e608a1111402c05fd63, user_id=dbc44a15b5594b599f61ab1e41244619, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:43.102540 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a4288-5ce9-4e91-b46f-c752c8cc60b3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:43.115817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1222/2438] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:43 2022] PUT /identity/v3/projects/c6ea5cb981164e608a1111402c05fd63/users/dbc44a15b5594b599f61ab1e41244619/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:43.119827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f26629-2777-411a-b7b7-29f735c38c30 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:43.130386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f26629-2777-411a-b7b7-29f735c38c30 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:43.188324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:43.188997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.189312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.189594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.190320 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:43.192824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:43.206921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1217/2439] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:43.208095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a4288-5ce9-4e91-b46f-c752c8cc60b3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:43.218712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:43.219350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a4288-5ce9-4e91-b46f-c752c8cc60b3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:43.220526 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:43.224831 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875017f7-27a2-414c-9d55-60639a57b540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:43.225725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875017f7-27a2-414c-9d55-60639a57b540 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.226009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875017f7-27a2-414c-9d55-60639a57b540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.226289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875017f7-27a2-414c-9d55-60639a57b540 admin admin] PATH_INFO: `/v3/projects/c6ea5cb981164e608a1111402c05fd63/users/dbc44a15b5594b599f61ab1e41244619/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.228285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875017f7-27a2-414c-9d55-60639a57b540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ea5cb981164e608a1111402c05fd63, user_id=dbc44a15b5594b599f61ab1e41244619, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:43.231838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875017f7-27a2-414c-9d55-60639a57b540 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:43.233669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34db2a-e62c-4283-adb4-e2c88ea93950 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:43.239735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:43.240673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.240963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.241247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] PATH_INFO: `/v3/users/d53b540119174d7f88ce866e8821e4d8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.242628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53b540119174d7f88ce866e8821e4d8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:43.244005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1223/2440] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:43 2022] PUT /identity/v3/projects/c6ea5cb981164e608a1111402c05fd63/users/dbc44a15b5594b599f61ab1e41244619/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:43.247917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:43.297038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53b540119174d7f88ce866e8821e4d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.309994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53b540119174d7f88ce866e8821e4d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.310784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53b540119174d7f88ce866e8821e4d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.318300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53b540119174d7f88ce866e8821e4d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.318995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53b540119174d7f88ce866e8821e4d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.333774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1218/2441] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:43 2022] DELETE /identity/v3/users/d53b540119174d7f88ce866e8821e4d8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:10:43.338462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dbeb9-02f0-431a-aacb-e58abfae8ef3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.338852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dbeb9-02f0-431a-aacb-e58abfae8ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.339145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dbeb9-02f0-431a-aacb-e58abfae8ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.363380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5f3dbeb9-02f0-431a-aacb-e58abfae8ef3 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:43.366183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f3dbeb9-02f0-431a-aacb-e58abfae8ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:43.425995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:43.441556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1224/2442] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:43.452181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:43.453450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:43.514985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240685e0-34cc-40e0-bcf0-f9382d23eddc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:43.519604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d564c67-ef12-46d9-9666-a5bca2eecbd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:43.520507 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d564c67-ef12-46d9-9666-a5bca2eecbd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.521165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d564c67-ef12-46d9-9666-a5bca2eecbd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.521165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d564c67-ef12-46d9-9666-a5bca2eecbd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.532726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d564c67-ef12-46d9-9666-a5bca2eecbd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:43.534207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d564c67-ef12-46d9-9666-a5bca2eecbd3 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:43.539423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:43.548262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1219/2443] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:43.870466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dbeb9-02f0-431a-aacb-e58abfae8ef3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:43.872469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f3dbeb9-02f0-431a-aacb-e58abfae8ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:43.889922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dbeb9-02f0-431a-aacb-e58abfae8ef3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:43.896197 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:43.897095 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:43.897385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:43.897667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] PATH_INFO: `/v3/projects/32006abfeddd41f5ba0643dd50a3d4e2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:43.899329 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32006abfeddd41f5ba0643dd50a3d4e2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:43.901260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:43.961632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32006abfeddd41f5ba0643dd50a3d4e2', 'request_id': 'req-85a961f2-ac9b-450f-9cb3-a93805bece32'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.973346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32006abfeddd41f5ba0643dd50a3d4e2', 'request_id': 'req-85a961f2-ac9b-450f-9cb3-a93805bece32'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.980072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32006abfeddd41f5ba0643dd50a3d4e2', 'request_id': 'req-85a961f2-ac9b-450f-9cb3-a93805bece32'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.992378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] The token cache is being invalidate because project 32006abfeddd41f5ba0643dd50a3d4e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:43.992748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.993380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-85a961f2-ac9b-450f-9cb3-a93805bece32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:43.994975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1225/2444] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:43 2022] DELETE /identity/v3/projects/32006abfeddd41f5ba0643dd50a3d4e2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:43.998947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d564c67-ef12-46d9-9666-a5bca2eecbd3 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:44.010152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d564c67-ef12-46d9-9666-a5bca2eecbd3 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:44.087270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:44.088658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:44.089082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:44.089498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] PATH_INFO: `/v3/users/85f4a01fb408423382c312f6a95d360d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:44.100022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f4a01fb408423382c312f6a95d360d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:44.100022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:44.101577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af4f9-cd94-4475-9587-02051f37b368 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:44.101878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af4f9-cd94-4475-9587-02051f37b368 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:44.102170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64af4f9-cd94-4475-9587-02051f37b368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:44.140909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c64af4f9-cd94-4475-9587-02051f37b368 None None] MFA Rules not processed for user `dbc44a15b5594b599f61ab1e41244619`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:44.142143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c64af4f9-cd94-4475-9587-02051f37b368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:44.169390 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f4a01fb408423382c312f6a95d360d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.185681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f4a01fb408423382c312f6a95d360d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.186444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f4a01fb408423382c312f6a95d360d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.197197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f4a01fb408423382c312f6a95d360d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.197915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f4a01fb408423382c312f6a95d360d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.217839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1220/2445] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:43 2022] DELETE /identity/v3/users/85f4a01fb408423382c312f6a95d360d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:44.275849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1226/2446] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:10:44.289083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:44.291712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:44.296116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b1987-4f33-461b-8271-b3503ee722ea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:44.299156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b1987-4f33-461b-8271-b3503ee722ea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:44.299886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b1987-4f33-461b-8271-b3503ee722ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:44.324856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-406b1987-4f33-461b-8271-b3503ee722ea None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:44.326331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-406b1987-4f33-461b-8271-b3503ee722ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:44.380852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f70d4-aa45-47c7-afc9-e0bf28054678 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:44.385186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:44.386729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:44.386896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:44.387467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:44.390645 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:44.395711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1227/2447] 10.222.0.32 () {58 vars in 967 bytes} [Mon May 30 12:10:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:44.408263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b1987-4f33-461b-8271-b3503ee722ea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:44.409971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-406b1987-4f33-461b-8271-b3503ee722ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:44.426907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:44.441930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:44.478450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:44.479913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:44.491794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1221/2448] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:44.514742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b1987-4f33-461b-8271-b3503ee722ea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:44.522349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2681fd-115d-4b51-9303-533026701ac3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:44.523021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2681fd-115d-4b51-9303-533026701ac3 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:44.523308 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2681fd-115d-4b51-9303-533026701ac3 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:44.523588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2681fd-115d-4b51-9303-533026701ac3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:44.533987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2681fd-115d-4b51-9303-533026701ac3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:44.536351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2681fd-115d-4b51-9303-533026701ac3 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:44.545040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1228/2449] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:10:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 30 12:10:44.617171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:44.617171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:44.716601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1dea2-c04b-4fbb-b9bc-b0e4d12e58e4 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:44.725304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:44.726285 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:44.726560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:44.726862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] PATH_INFO: `/v3/projects/ef31b84e0834498c9a3390710e68375c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:44.728920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef31b84e0834498c9a3390710e68375c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:44.730874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:44.835711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef31b84e0834498c9a3390710e68375c', 'request_id': 'req-78e90dc8-2f4b-493c-9433-51b6460beee3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.852216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef31b84e0834498c9a3390710e68375c', 'request_id': 'req-78e90dc8-2f4b-493c-9433-51b6460beee3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.860818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef31b84e0834498c9a3390710e68375c', 'request_id': 'req-78e90dc8-2f4b-493c-9433-51b6460beee3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.878599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] The token cache is being invalidate because project ef31b84e0834498c9a3390710e68375c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:44.879116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.880048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-78e90dc8-2f4b-493c-9433-51b6460beee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:44.882868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1222/2450] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:44 2022] DELETE /identity/v3/projects/ef31b84e0834498c9a3390710e68375c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:44.887978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88738-8616-4b34-a835-bcb443879e5e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:44.888548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88738-8616-4b34-a835-bcb443879e5e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:44.888650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88738-8616-4b34-a835-bcb443879e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:44.919917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-33d88738-8616-4b34-a835-bcb443879e5e None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:44.921745 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-33d88738-8616-4b34-a835-bcb443879e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:44.997463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1229/2451] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:45.005975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efd47-3780-4fd2-b1d6-67a97c1e36ce None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.006486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efd47-3780-4fd2-b1d6-67a97c1e36ce None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.006618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efd47-3780-4fd2-b1d6-67a97c1e36ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.027199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e09efd47-3780-4fd2-b1d6-67a97c1e36ce None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:45.028460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e09efd47-3780-4fd2-b1d6-67a97c1e36ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:45.037078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1223/2452] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:45.041232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d88738-8616-4b34-a835-bcb443879e5e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.052966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d88738-8616-4b34-a835-bcb443879e5e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.058622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.059300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.059586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.059865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.060766 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:10:45.061097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.062726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:45.079594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1230/2453] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:10:45.080625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efd47-3780-4fd2-b1d6-67a97c1e36ce None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.090537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efd47-3780-4fd2-b1d6-67a97c1e36ce None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.095761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876c589-ac02-4c50-8a60-1b7c4752b286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.096471 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876c589-ac02-4c50-8a60-1b7c4752b286 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.096756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876c589-ac02-4c50-8a60-1b7c4752b286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.097214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876c589-ac02-4c50-8a60-1b7c4752b286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.097914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876c589-ac02-4c50-8a60-1b7c4752b286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.099842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876c589-ac02-4c50-8a60-1b7c4752b286 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:45.131413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1224/2454] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:45 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:45.135585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.147717 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a20e58-465e-4381-97a8-463070a7743d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.216313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a16bc-08d3-4b04-9f2e-3fefcc3c729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.217556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a16bc-08d3-4b04-9f2e-3fefcc3c729e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.217556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a16bc-08d3-4b04-9f2e-3fefcc3c729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.217788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a16bc-08d3-4b04-9f2e-3fefcc3c729e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.218679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a16bc-08d3-4b04-9f2e-3fefcc3c729e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.220662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a16bc-08d3-4b04-9f2e-3fefcc3c729e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:45.247927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1231/2455] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:45 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:45.249126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876c589-ac02-4c50-8a60-1b7c4752b286 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.263092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876c589-ac02-4c50-8a60-1b7c4752b286 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.270544 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d356d59-6c91-4f26-95d0-9328b4372d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.271223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356d59-6c91-4f26-95d0-9328b4372d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.271508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356d59-6c91-4f26-95d0-9328b4372d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.271781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356d59-6c91-4f26-95d0-9328b4372d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.272568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d356d59-6c91-4f26-95d0-9328b4372d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.274189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d356d59-6c91-4f26-95d0-9328b4372d2f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:45.293454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1225/2456] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:45.294666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a16bc-08d3-4b04-9f2e-3fefcc3c729e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.305492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a16bc-08d3-4b04-9f2e-3fefcc3c729e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.310705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7832a7d-a196-4076-8839-f67e3e3abaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.311385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7832a7d-a196-4076-8839-f67e3e3abaed admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.311672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7832a7d-a196-4076-8839-f67e3e3abaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.311951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7832a7d-a196-4076-8839-f67e3e3abaed admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.312709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7832a7d-a196-4076-8839-f67e3e3abaed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.314300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7832a7d-a196-4076-8839-f67e3e3abaed admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:45.329625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1232/2457] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:45.334143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d356d59-6c91-4f26-95d0-9328b4372d2f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.344437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d356d59-6c91-4f26-95d0-9328b4372d2f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.349689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.350586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.350872 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.351155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] PATH_INFO: `/v3/projects/0eaef0b115a0406483dcca42f4aec0ae/users/a733dd5f437b47149828218d690a3dfb/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.368092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eaef0b115a0406483dcca42f4aec0ae, user_id=a733dd5f437b47149828218d690a3dfb, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.370826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:45.390487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:45.392606 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1226/2458] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:45 2022] PUT /identity/v3/projects/0eaef0b115a0406483dcca42f4aec0ae/users/a733dd5f437b47149828218d690a3dfb/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:45.403075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276387a2-eb18-4a7f-8bdc-227c096a48e9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.403372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276387a2-eb18-4a7f-8bdc-227c096a48e9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.403659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276387a2-eb18-4a7f-8bdc-227c096a48e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.411525 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:45.435658 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-276387a2-eb18-4a7f-8bdc-227c096a48e9 None None] MFA Rules not processed for user `a733dd5f437b47149828218d690a3dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:45.437225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-276387a2-eb18-4a7f-8bdc-227c096a48e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:45.512475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1233/2459] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:45.524523 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.525245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:45.598054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac7287-1cae-4136-8c6d-b39d3890eed4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.603099 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9044cff-dc0e-4b8d-a7d7-f61d8b79de6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.603706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9044cff-dc0e-4b8d-a7d7-f61d8b79de6b service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.604028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9044cff-dc0e-4b8d-a7d7-f61d8b79de6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.604375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9044cff-dc0e-4b8d-a7d7-f61d8b79de6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.617007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9044cff-dc0e-4b8d-a7d7-f61d8b79de6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.618917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9044cff-dc0e-4b8d-a7d7-f61d8b79de6b service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:45.630007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1227/2460] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:45.652374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276387a2-eb18-4a7f-8bdc-227c096a48e9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.719989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276387a2-eb18-4a7f-8bdc-227c096a48e9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.724992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.725906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.726194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.726495 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] PATH_INFO: `/v3/users/a733dd5f437b47149828218d690a3dfb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.727741 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a733dd5f437b47149828218d690a3dfb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.729787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:45.778839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a733dd5f437b47149828218d690a3dfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:45.790530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a733dd5f437b47149828218d690a3dfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:45.791227 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a733dd5f437b47149828218d690a3dfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:45.791826 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a733dd5f437b47149828218d690a3dfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:45.798511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a733dd5f437b47149828218d690a3dfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:45.811794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1234/2461] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:10:45 2022] DELETE /identity/v3/users/a733dd5f437b47149828218d690a3dfb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:45.816983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e21e3-8609-49d9-836d-a60f6751a5f2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.817273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e21e3-8609-49d9-836d-a60f6751a5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.817553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e21e3-8609-49d9-836d-a60f6751a5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.834612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-007e21e3-8609-49d9-836d-a60f6751a5f2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:45.835835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-007e21e3-8609-49d9-836d-a60f6751a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:45.896049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1228/2462] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:45.905714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.907173 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:45.922970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:45.976692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ad86-61be-474d-ba03-a66867032d15 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:45.980618 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:45.981143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e21e3-8609-49d9-836d-a60f6751a5f2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:45.981362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:45.981656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:45.981941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:45.982730 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-007e21e3-8609-49d9-836d-a60f6751a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:45.993144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:45.994906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.006462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1235/2463] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:46.048755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e21e3-8609-49d9-836d-a60f6751a5f2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:46.053436 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8f694-04fc-4da8-8753-5c908b64238d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:46.054420 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8f694-04fc-4da8-8753-5c908b64238d service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.054938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8f694-04fc-4da8-8753-5c908b64238d service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.055522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8f694-04fc-4da8-8753-5c908b64238d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.058554 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-11c8f694-04fc-4da8-8753-5c908b64238d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:46.133630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:46.147954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8f694-04fc-4da8-8753-5c908b64238d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:46.149440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8f694-04fc-4da8-8753-5c908b64238d service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.160728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1229/2464] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:46.407126 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:46.409145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:46.447733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e20982-76df-4224-9276-eea15db32dfe service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:46.452405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:46.453277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.453560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.453833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] PATH_INFO: `/v3/projects/0eaef0b115a0406483dcca42f4aec0ae` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.455034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eaef0b115a0406483dcca42f4aec0ae)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:46.457553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.506733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eaef0b115a0406483dcca42f4aec0ae', 'request_id': 'req-67467343-2d1b-4a0d-b16e-5d89bf5af07d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:46.514316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eaef0b115a0406483dcca42f4aec0ae', 'request_id': 'req-67467343-2d1b-4a0d-b16e-5d89bf5af07d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:46.521255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eaef0b115a0406483dcca42f4aec0ae', 'request_id': 'req-67467343-2d1b-4a0d-b16e-5d89bf5af07d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:46.531672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] The token cache is being invalidate because project 0eaef0b115a0406483dcca42f4aec0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:10:46.531969 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:46.532775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-67467343-2d1b-4a0d-b16e-5d89bf5af07d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:10:46.534526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1236/2465] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:10:46 2022] DELETE /identity/v3/projects/0eaef0b115a0406483dcca42f4aec0ae => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:46.539752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf28f2-e0de-48d9-be12-032758a4caaf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.540047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf28f2-e0de-48d9-be12-032758a4caaf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.540363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf28f2-e0de-48d9-be12-032758a4caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.559591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9acf28f2-e0de-48d9-be12-032758a4caaf None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:46.560912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9acf28f2-e0de-48d9-be12-032758a4caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:46.630807 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1230/2466] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:46.639715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab40eba-b9bd-40a0-b15f-6c89986d5d3b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.640036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab40eba-b9bd-40a0-b15f-6c89986d5d3b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.640360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab40eba-b9bd-40a0-b15f-6c89986d5d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.659498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8ab40eba-b9bd-40a0-b15f-6c89986d5d3b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:46.660916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ab40eba-b9bd-40a0-b15f-6c89986d5d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:46.669357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1237/2467] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:46.673640 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf28f2-e0de-48d9-be12-032758a4caaf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:46.684210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf28f2-e0de-48d9-be12-032758a4caaf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:46.688975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:46.689635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.689920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.690206 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.691735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:10:46.692054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:46.693827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.706943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1231/2468] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:10:46.711389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab40eba-b9bd-40a0-b15f-6c89986d5d3b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:46.720548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab40eba-b9bd-40a0-b15f-6c89986d5d3b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:46.726381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0e7e-a18e-4f06-98a4-10aaedb34ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:46.727238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0e7e-a18e-4f06-98a4-10aaedb34ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.727638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0e7e-a18e-4f06-98a4-10aaedb34ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.727978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0e7e-a18e-4f06-98a4-10aaedb34ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.728867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf0e7e-a18e-4f06-98a4-10aaedb34ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:46.730788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf0e7e-a18e-4f06-98a4-10aaedb34ff2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.767761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1238/2469] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:46 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:46.769473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:46.780280 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab027-e5a8-42b2-aa43-3b9a1eae37c7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:46.840623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7144d1-583b-41d2-a74d-e2a803655a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:46.841226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7144d1-583b-41d2-a74d-e2a803655a66 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.841509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7144d1-583b-41d2-a74d-e2a803655a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.841786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7144d1-583b-41d2-a74d-e2a803655a66 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.842477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7144d1-583b-41d2-a74d-e2a803655a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:46.844426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7144d1-583b-41d2-a74d-e2a803655a66 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.869504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1232/2470] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:46 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:46.870666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0e7e-a18e-4f06-98a4-10aaedb34ff2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:46.884587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0e7e-a18e-4f06-98a4-10aaedb34ff2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:46.889752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b24b-fc17-4b99-aa34-1151ab42caf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:46.890528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b24b-fc17-4b99-aa34-1151ab42caf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.890715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b24b-fc17-4b99-aa34-1151ab42caf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.890974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b24b-fc17-4b99-aa34-1151ab42caf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.891703 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8b24b-fc17-4b99-aa34-1151ab42caf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:46.893656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8b24b-fc17-4b99-aa34-1151ab42caf8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.905235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1239/2471] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:46.908900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7144d1-583b-41d2-a74d-e2a803655a66 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:46.919839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7144d1-583b-41d2-a74d-e2a803655a66 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:46.924635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579eac8a-3c92-4e73-93a7-62ab3f701684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:46.925406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eac8a-3c92-4e73-93a7-62ab3f701684 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.925684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eac8a-3c92-4e73-93a7-62ab3f701684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.925952 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579eac8a-3c92-4e73-93a7-62ab3f701684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.926678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579eac8a-3c92-4e73-93a7-62ab3f701684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:46.928272 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579eac8a-3c92-4e73-93a7-62ab3f701684 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.938921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1233/2472] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:10:46.945140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b24b-fc17-4b99-aa34-1151ab42caf8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:46.955203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b24b-fc17-4b99-aa34-1151ab42caf8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:46.960289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:46.961205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:46.961486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:46.961766 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] PATH_INFO: `/v3/projects/dc19bd60cde244559a823a47293540f7/users/c8915c74c4d24d2cb1cded5aa7927120/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:46.978958 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc19bd60cde244559a823a47293540f7, user_id=c8915c74c4d24d2cb1cded5aa7927120, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:46.981686 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:46.996945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1240/2473] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:46 2022] PUT /identity/v3/projects/dc19bd60cde244559a823a47293540f7/users/c8915c74c4d24d2cb1cded5aa7927120/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:47.003959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d36a-fafa-41b4-83d5-baa7db97b700 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:47.004302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d36a-fafa-41b4-83d5-baa7db97b700 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:47.004592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d36a-fafa-41b4-83d5-baa7db97b700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:47.023433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1516d36a-fafa-41b4-83d5-baa7db97b700 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:47.024828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1516d36a-fafa-41b4-83d5-baa7db97b700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:47.082190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:47.095042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1234/2474] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:10:47.106731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:47.108500 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:47.156426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:47.175722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de99e43-deb6-422a-a16a-b3fa7e30965d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:47.179510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465070f-f353-4efd-85de-1b56ce1c66e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:47.180173 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465070f-f353-4efd-85de-1b56ce1c66e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:47.180472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465070f-f353-4efd-85de-1b56ce1c66e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:47.180753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465070f-f353-4efd-85de-1b56ce1c66e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:47.189709 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465070f-f353-4efd-85de-1b56ce1c66e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:47.191442 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465070f-f353-4efd-85de-1b56ce1c66e9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:47.200835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1241/2475] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:52.458647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b7a6c-bc40-4087-9a04-2448e8026fc8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:52.459192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b7a6c-bc40-4087-9a04-2448e8026fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:52.459192 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b7a6c-bc40-4087-9a04-2448e8026fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:52.488553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-206b7a6c-bc40-4087-9a04-2448e8026fc8 None None] MFA Rules not processed for user `c8915c74c4d24d2cb1cded5aa7927120`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:52.489800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-206b7a6c-bc40-4087-9a04-2448e8026fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:52.569813 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1235/2476] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:52.574544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465070f-f353-4efd-85de-1b56ce1c66e9 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:52.584803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465070f-f353-4efd-85de-1b56ce1c66e9 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:52.593310 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0676c05-6494-407a-9527-a79999063a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:52.594032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0676c05-6494-407a-9527-a79999063a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:52.594715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0676c05-6494-407a-9527-a79999063a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:52.594715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0676c05-6494-407a-9527-a79999063a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:52.595275 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0676c05-6494-407a-9527-a79999063a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:52.597824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0676c05-6494-407a-9527-a79999063a0c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:52.627045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1242/2477] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:10:52 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:52.630817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b7a6c-bc40-4087-9a04-2448e8026fc8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:52.642486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b7a6c-bc40-4087-9a04-2448e8026fc8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:52.706982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb71e9c-f9fa-44da-897a-0980dc59ba45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:52.707677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb71e9c-f9fa-44da-897a-0980dc59ba45 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:52.707964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb71e9c-f9fa-44da-897a-0980dc59ba45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:52.708290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb71e9c-f9fa-44da-897a-0980dc59ba45 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:52.709004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb71e9c-f9fa-44da-897a-0980dc59ba45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:52.710977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb71e9c-f9fa-44da-897a-0980dc59ba45 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:52.733274 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1236/2478] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:10:52 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:10:52.737326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0676c05-6494-407a-9527-a79999063a0c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:52.748297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0676c05-6494-407a-9527-a79999063a0c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:52.754156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d017c4-228f-4032-82a0-113bb65347ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:52.754930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017c4-228f-4032-82a0-113bb65347ee admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:52.755303 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017c4-228f-4032-82a0-113bb65347ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:52.755662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017c4-228f-4032-82a0-113bb65347ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:52.756531 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d017c4-228f-4032-82a0-113bb65347ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:52.758314 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d017c4-228f-4032-82a0-113bb65347ee admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:52.772624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1243/2479] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:10:52.777191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb71e9c-f9fa-44da-897a-0980dc59ba45 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:52.801250 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb71e9c-f9fa-44da-897a-0980dc59ba45 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:52.807634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737a787-1185-4986-8897-dd3360ef571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:52.809481 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737a787-1185-4986-8897-dd3360ef571d admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:52.810072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737a787-1185-4986-8897-dd3360ef571d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:52.810672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737a787-1185-4986-8897-dd3360ef571d admin admin] PATH_INFO: `/v3/projects/20c153c20fd24882972be249a215ac2c/users/02202bbdf9014e0284db9f6ab61d1480/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:52.831570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737a787-1185-4986-8897-dd3360ef571d admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c153c20fd24882972be249a215ac2c, user_id=02202bbdf9014e0284db9f6ab61d1480, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:52.834932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737a787-1185-4986-8897-dd3360ef571d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:52.852448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1237/2480] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:52 2022] PUT /identity/v3/projects/20c153c20fd24882972be249a215ac2c/users/02202bbdf9014e0284db9f6ab61d1480/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:52.856066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d017c4-228f-4032-82a0-113bb65347ee admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:52.871831 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d017c4-228f-4032-82a0-113bb65347ee admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:52.949325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253dd15-aee4-4149-8660-a515953e7260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:52.950120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253dd15-aee4-4149-8660-a515953e7260 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:52.950449 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253dd15-aee4-4149-8660-a515953e7260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:52.950806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253dd15-aee4-4149-8660-a515953e7260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:52.951557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253dd15-aee4-4149-8660-a515953e7260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:52.953280 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253dd15-aee4-4149-8660-a515953e7260 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:52.967775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1244/2481] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:10:52.971032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737a787-1185-4986-8897-dd3360ef571d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:52.984439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737a787-1185-4986-8897-dd3360ef571d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:52.990329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e85fa-0e4b-4958-9659-65b46ef8c66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:52.991228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e85fa-0e4b-4958-9659-65b46ef8c66c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:52.991519 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e85fa-0e4b-4958-9659-65b46ef8c66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:52.991799 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e85fa-0e4b-4958-9659-65b46ef8c66c admin admin] PATH_INFO: `/v3/projects/20c153c20fd24882972be249a215ac2c/users/02202bbdf9014e0284db9f6ab61d1480/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:52.994059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e85fa-0e4b-4958-9659-65b46ef8c66c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c153c20fd24882972be249a215ac2c, user_id=02202bbdf9014e0284db9f6ab61d1480, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:52.996688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e85fa-0e4b-4958-9659-65b46ef8c66c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:53.009419 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1238/2482] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:10:52 2022] PUT /identity/v3/projects/20c153c20fd24882972be249a215ac2c/users/02202bbdf9014e0284db9f6ab61d1480/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:10:55.563788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253dd15-aee4-4149-8660-a515953e7260 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:55.650815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253dd15-aee4-4149-8660-a515953e7260 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:55.788221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:55.788911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:55.789199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:55.789482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:55.792931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:55.808915 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:55.980722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:55.984117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:56.008438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1245/2483] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:10:59.435754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402eb2ed-2c16-41bb-8da0-1935b1e6563f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:59.436966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402eb2ed-2c16-41bb-8da0-1935b1e6563f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:59.436966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402eb2ed-2c16-41bb-8da0-1935b1e6563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:59.472537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-402eb2ed-2c16-41bb-8da0-1935b1e6563f None None] MFA Rules not processed for user `02202bbdf9014e0284db9f6ab61d1480`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:10:59.474144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-402eb2ed-2c16-41bb-8da0-1935b1e6563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:59.515553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:10:59.575881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1239/2484] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:10:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:10:59.588685 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:10:59.590153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:10:59.779301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edd4e7-4b2b-4693-9235-4f7d9b2e121c service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:10:59.789943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad37bdce39f64836b0cd14613cabe91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:10:59.790639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:10:59.791068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:10:59.791357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:10:59.941846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:10:59.943510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:10:59.973234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1246/2485] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:10:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:00.176423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb0466-461d-4d1f-984d-e54baffd9178 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:00.177959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb0466-461d-4d1f-984d-e54baffd9178 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:00.178859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb0466-461d-4d1f-984d-e54baffd9178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:00.212879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-50cb0466-461d-4d1f-984d-e54baffd9178 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:00.214293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-50cb0466-461d-4d1f-984d-e54baffd9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:00.305905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1240/2486] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:11:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:00.324870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:00.326516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:00.388357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:00.420423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd29821-0a63-4b23-9f4c-604a3d4f4a60 service glance] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:00.426496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cb97-585b-4ad3-8ee4-e42154a62039 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:00.427099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cb97-585b-4ad3-8ee4-e42154a62039 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:00.427425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cb97-585b-4ad3-8ee4-e42154a62039 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:00.427671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635cb97-585b-4ad3-8ee4-e42154a62039 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:00.437696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635cb97-585b-4ad3-8ee4-e42154a62039 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:00.439978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635cb97-585b-4ad3-8ee4-e42154a62039 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:00.449486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1247/2487] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:11:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:11:01.619282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb0466-461d-4d1f-984d-e54baffd9178 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:01.621444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-50cb0466-461d-4d1f-984d-e54baffd9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:01.646703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb0466-461d-4d1f-984d-e54baffd9178 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:01.657116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:01.659757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:01.661884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:01.663509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] PATH_INFO: `/v3/users/39a41a7b2ea64ca3ad13e70a83bf28ca` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:01.678585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a41a7b2ea64ca3ad13e70a83bf28ca)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:01.700904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:01.828438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a41a7b2ea64ca3ad13e70a83bf28ca'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:01.854076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a41a7b2ea64ca3ad13e70a83bf28ca'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:01.855212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a41a7b2ea64ca3ad13e70a83bf28ca'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:01.866371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a41a7b2ea64ca3ad13e70a83bf28ca'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:01.867622 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a41a7b2ea64ca3ad13e70a83bf28ca'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:01.916908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1241/2488] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:01 2022] DELETE /identity/v3/users/39a41a7b2ea64ca3ad13e70a83bf28ca => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:02.233407 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cb97-585b-4ad3-8ee4-e42154a62039 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:02.251155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635cb97-585b-4ad3-8ee4-e42154a62039 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:02.352656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:02.353647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:02.354007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:02.354357 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] PATH_INFO: `/v3/projects/5b976277ecde412ab4b17b2d17f14340` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:02.355992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b976277ecde412ab4b17b2d17f14340)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:02.358842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:02.436271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b976277ecde412ab4b17b2d17f14340', 'request_id': 'req-fbc47e1f-ab14-4516-b33e-141883c999a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.446305 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b976277ecde412ab4b17b2d17f14340', 'request_id': 'req-fbc47e1f-ab14-4516-b33e-141883c999a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.454939 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b976277ecde412ab4b17b2d17f14340', 'request_id': 'req-fbc47e1f-ab14-4516-b33e-141883c999a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.468997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] The token cache is being invalidate because project 5b976277ecde412ab4b17b2d17f14340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:02.469290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.470017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.473308 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1248/2489] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:02 2022] DELETE /identity/v3/projects/5b976277ecde412ab4b17b2d17f14340 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:02.477123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:02.479497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:02.500274 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97cfb-dbd7-4252-9009-ff2af086e8b4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:02.585615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:02.628229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:02.631276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:02.635161 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:02.636075 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:02.642012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:02.645219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] PATH_INFO: `/v3/users/dbc44a15b5594b599f61ab1e41244619` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:02.652551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc44a15b5594b599f61ab1e41244619)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:02.654780 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:02.667622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc47e1f-ab14-4516-b33e-141883c999a0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:02.681098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:02.689492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:02.689492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:02.689869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] PATH_INFO: `/v3/users/7575179edbda45208561c983fc5b956c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:02.694033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7575179edbda45208561c983fc5b956c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:02.703715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:02.733993 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc44a15b5594b599f61ab1e41244619'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.751894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc44a15b5594b599f61ab1e41244619'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.752776 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc44a15b5594b599f61ab1e41244619'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.762182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc44a15b5594b599f61ab1e41244619'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.762936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc44a15b5594b599f61ab1e41244619'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.797212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1242/2490] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:02 2022] DELETE /identity/v3/users/dbc44a15b5594b599f61ab1e41244619 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:02.849536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7575179edbda45208561c983fc5b956c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.863965 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:02.884585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7575179edbda45208561c983fc5b956c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.885389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7575179edbda45208561c983fc5b956c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.886987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7575179edbda45208561c983fc5b956c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.905406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7575179edbda45208561c983fc5b956c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:02.959813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1249/2491] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:02 2022] DELETE /identity/v3/users/7575179edbda45208561c983fc5b956c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:03.178820 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:03.181855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:03.285506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:03.287068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:03.325771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a3a7b-99dc-4ecf-8453-8ecc6c642803 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:03.397275 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c03237-3576-4551-a047-300c3b119e15 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:03.403457 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:03.404454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.404803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.405185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] PATH_INFO: `/v3/projects/20763921c37c4e81bba702e09bf628b2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.406649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20763921c37c4e81bba702e09bf628b2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:03.409167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:03.440281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:03.441166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.441455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.441740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] PATH_INFO: `/v3/projects/c6ea5cb981164e608a1111402c05fd63` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.443406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ea5cb981164e608a1111402c05fd63)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:03.446144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:03.469483 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:03.478755 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20763921c37c4e81bba702e09bf628b2', 'request_id': 'req-03ba14bf-1f90-4d6b-a90e-dfd882006672'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.493058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20763921c37c4e81bba702e09bf628b2', 'request_id': 'req-03ba14bf-1f90-4d6b-a90e-dfd882006672'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.505144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20763921c37c4e81bba702e09bf628b2', 'request_id': 'req-03ba14bf-1f90-4d6b-a90e-dfd882006672'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.522232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] The token cache is being invalidate because project 20763921c37c4e81bba702e09bf628b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:03.523003 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.524493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-03ba14bf-1f90-4d6b-a90e-dfd882006672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.526927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1250/2492] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:03 2022] DELETE /identity/v3/projects/20763921c37c4e81bba702e09bf628b2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:03.534990 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356d9d6-cd6d-4228-a18e-4314810debb5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.535209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ea5cb981164e608a1111402c05fd63', 'request_id': 'req-a8625ef3-d82f-4b2c-8da3-105229efdd83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.539166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356d9d6-cd6d-4228-a18e-4314810debb5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.540239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356d9d6-cd6d-4228-a18e-4314810debb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.543465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ea5cb981164e608a1111402c05fd63', 'request_id': 'req-a8625ef3-d82f-4b2c-8da3-105229efdd83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.562299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ea5cb981164e608a1111402c05fd63', 'request_id': 'req-a8625ef3-d82f-4b2c-8da3-105229efdd83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.591108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9356d9d6-cd6d-4228-a18e-4314810debb5 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:03.595127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9356d9d6-cd6d-4228-a18e-4314810debb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:03.596513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] The token cache is being invalidate because project c6ea5cb981164e608a1111402c05fd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:03.596882 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.598161 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a8625ef3-d82f-4b2c-8da3-105229efdd83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:03.605456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1243/2493] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:03 2022] DELETE /identity/v3/projects/c6ea5cb981164e608a1111402c05fd63 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:03.608357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675037d-4cce-4b17-9dea-8750e22163b2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.608720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675037d-4cce-4b17-9dea-8750e22163b2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.609015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675037d-4cce-4b17-9dea-8750e22163b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.644638 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e675037d-4cce-4b17-9dea-8750e22163b2 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:03.646018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e675037d-4cce-4b17-9dea-8750e22163b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:03.708046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1251/2494] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:03.716145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797981f-313a-4f41-8589-703e7ae36186 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.716885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797981f-313a-4f41-8589-703e7ae36186 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.717410 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797981f-313a-4f41-8589-703e7ae36186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.730266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:03.765333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8797981f-313a-4f41-8589-703e7ae36186 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:03.766835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8797981f-313a-4f41-8589-703e7ae36186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:03.801433 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1252/2495] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:03.805317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797981f-313a-4f41-8589-703e7ae36186 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:03.835391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1244/2496] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:03.835631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797981f-313a-4f41-8589-703e7ae36186 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:03.840831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ace2d-472b-438f-b3ce-2da522b42fca None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.841403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ace2d-472b-438f-b3ce-2da522b42fca None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.841917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ace2d-472b-438f-b3ce-2da522b42fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.844952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:03.845613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.846456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.846850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.849273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:03.849713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:03.857851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:03.882482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-248ace2d-472b-438f-b3ce-2da522b42fca None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:03.884666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-248ace2d-472b-438f-b3ce-2da522b42fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:03.893578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1253/2497] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 30 12:11:03.897856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:03.906560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1245/2498] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:03.907082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ace2d-472b-438f-b3ce-2da522b42fca None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:03.911553 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7740781-20d5-41c0-a3bf-1ab22ce8e727 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:03.917157 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71245310-e27e-43d0-83ed-a5e481eea841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:03.918185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71245310-e27e-43d0-83ed-a5e481eea841 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.918326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71245310-e27e-43d0-83ed-a5e481eea841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.918729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71245310-e27e-43d0-83ed-a5e481eea841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.919779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71245310-e27e-43d0-83ed-a5e481eea841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:03.921989 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71245310-e27e-43d0-83ed-a5e481eea841 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:03.923139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ace2d-472b-438f-b3ce-2da522b42fca None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:03.934223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:03.935073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:03.935892 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:03.936209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:03.936962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:03.937273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:03.939055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:03.961905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1246/2499] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:03.970905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:03.981020 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1254/2500] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:03 2022] POST /identity/v3/projects => generated 405 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:03.983156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7d239-0dc1-4fe4-96ec-914aa6f78906 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:03.985616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71245310-e27e-43d0-83ed-a5e481eea841 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.103072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197fa95-79a3-432d-ad94-da0c534827ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.104354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197fa95-79a3-432d-ad94-da0c534827ee admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.104817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197fa95-79a3-432d-ad94-da0c534827ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.105258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197fa95-79a3-432d-ad94-da0c534827ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.106413 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197fa95-79a3-432d-ad94-da0c534827ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.109835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197fa95-79a3-432d-ad94-da0c534827ee admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.127058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71245310-e27e-43d0-83ed-a5e481eea841 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.133002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270026-9bd0-479b-bb46-5af74ea999f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.133690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270026-9bd0-479b-bb46-5af74ea999f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.133979 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270026-9bd0-479b-bb46-5af74ea999f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.134259 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92270026-9bd0-479b-bb46-5af74ea999f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.134955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270026-9bd0-479b-bb46-5af74ea999f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.137186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92270026-9bd0-479b-bb46-5af74ea999f7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.161054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1247/2501] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:03 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:04.165667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197fa95-79a3-432d-ad94-da0c534827ee admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.169366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1255/2502] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:03 2022] POST /identity/v3/users => generated 303 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:04.173359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:04.173547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270026-9bd0-479b-bb46-5af74ea999f7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.187394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92270026-9bd0-479b-bb46-5af74ea999f7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.280445 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197fa95-79a3-432d-ad94-da0c534827ee admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.293718 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1e4fd-28d2-466c-b98b-a24e6401cb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.294323 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1e4fd-28d2-466c-b98b-a24e6401cb56 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.295505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1e4fd-28d2-466c-b98b-a24e6401cb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.296678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1e4fd-28d2-466c-b98b-a24e6401cb56 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.297404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1e4fd-28d2-466c-b98b-a24e6401cb56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.299973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1e4fd-28d2-466c-b98b-a24e6401cb56 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.305387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aeee9-a2d0-40e0-bbcf-44a0f6503b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.306349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aeee9-a2d0-40e0-bbcf-44a0f6503b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.306646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aeee9-a2d0-40e0-bbcf-44a0f6503b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.306933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aeee9-a2d0-40e0-bbcf-44a0f6503b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.307685 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aeee9-a2d0-40e0-bbcf-44a0f6503b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.309432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aeee9-a2d0-40e0-bbcf-44a0f6503b45 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.334300 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1256/2503] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:04.341801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1248/2504] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:04 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:04.342012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aeee9-a2d0-40e0-bbcf-44a0f6503b45 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.344147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1e4fd-28d2-466c-b98b-a24e6401cb56 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.356372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aeee9-a2d0-40e0-bbcf-44a0f6503b45 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.362738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1e4fd-28d2-466c-b98b-a24e6401cb56 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.366036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e29a1f-a6f9-484e-8cc2-c90807fac967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.366919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e29a1f-a6f9-484e-8cc2-c90807fac967 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.367416 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e29a1f-a6f9-484e-8cc2-c90807fac967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.367862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e29a1f-a6f9-484e-8cc2-c90807fac967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.368894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e29a1f-a6f9-484e-8cc2-c90807fac967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.370688 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e29a1f-a6f9-484e-8cc2-c90807fac967 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.374648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117bc77-4483-406e-8269-4541c9d1312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.376017 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117bc77-4483-406e-8269-4541c9d1312a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.376416 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117bc77-4483-406e-8269-4541c9d1312a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.376774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117bc77-4483-406e-8269-4541c9d1312a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.384478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117bc77-4483-406e-8269-4541c9d1312a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.388200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117bc77-4483-406e-8269-4541c9d1312a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.389842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1257/2505] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:04.395261 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e29a1f-a6f9-484e-8cc2-c90807fac967 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.406609 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e29a1f-a6f9-484e-8cc2-c90807fac967 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.415329 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3dd96-8251-4b64-ab31-3d3f9d0bbfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.416340 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3dd96-8251-4b64-ab31-3d3f9d0bbfe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.416630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3dd96-8251-4b64-ab31-3d3f9d0bbfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.416919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3dd96-8251-4b64-ab31-3d3f9d0bbfe6 admin admin] PATH_INFO: `/v3/projects/b99ee7dfe9dc4e98a38d3463bc0464b4/users/062d003aac4e4521bc6c5bd8123d95d9/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.430888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1249/2506] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:04.436610 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117bc77-4483-406e-8269-4541c9d1312a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.446051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3dd96-8251-4b64-ab31-3d3f9d0bbfe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99ee7dfe9dc4e98a38d3463bc0464b4, user_id=062d003aac4e4521bc6c5bd8123d95d9, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.449447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3dd96-8251-4b64-ab31-3d3f9d0bbfe6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.454961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117bc77-4483-406e-8269-4541c9d1312a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.461887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713c4d4-81f2-4b03-b480-8f5bb8f1c524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.462621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713c4d4-81f2-4b03-b480-8f5bb8f1c524 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.462992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713c4d4-81f2-4b03-b480-8f5bb8f1c524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.463412 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713c4d4-81f2-4b03-b480-8f5bb8f1c524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.464457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713c4d4-81f2-4b03-b480-8f5bb8f1c524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.466448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713c4d4-81f2-4b03-b480-8f5bb8f1c524 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.480360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1258/2507] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:04 2022] PUT /identity/v3/projects/b99ee7dfe9dc4e98a38d3463bc0464b4/users/062d003aac4e4521bc6c5bd8123d95d9/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:04.481974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc33d4-1f20-4ff9-a495-a32adcefc1d6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.482338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc33d4-1f20-4ff9-a495-a32adcefc1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.482694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc33d4-1f20-4ff9-a495-a32adcefc1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.486965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1250/2508] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:04.489336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713c4d4-81f2-4b03-b480-8f5bb8f1c524 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.501269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713c4d4-81f2-4b03-b480-8f5bb8f1c524 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.513067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c7bc33d4-1f20-4ff9-a495-a32adcefc1d6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:04.514376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c7bc33d4-1f20-4ff9-a495-a32adcefc1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:04.532345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:04.618707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbf46c-10e3-491e-8445-ce0e8bca308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.620433 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbf46c-10e3-491e-8445-ce0e8bca308f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.621114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbf46c-10e3-491e-8445-ce0e8bca308f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.621970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbf46c-10e3-491e-8445-ce0e8bca308f admin admin] PATH_INFO: `/v3/projects/5f8b14b6ddb049b6ae4a02df41c1663a/users/ff204b48ac0d4eccb05e339c6adeaefe/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.647313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1259/2509] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:04.648125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbf46c-10e3-491e-8445-ce0e8bca308f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8b14b6ddb049b6ae4a02df41c1663a, user_id=ff204b48ac0d4eccb05e339c6adeaefe, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.657576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbf46c-10e3-491e-8445-ce0e8bca308f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.658297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc33d4-1f20-4ff9-a495-a32adcefc1d6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.660588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c7bc33d4-1f20-4ff9-a495-a32adcefc1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:04.684205 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1251/2510] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:04 2022] PUT /identity/v3/projects/5f8b14b6ddb049b6ae4a02df41c1663a/users/ff204b48ac0d4eccb05e339c6adeaefe/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:04.754038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc33d4-1f20-4ff9-a495-a32adcefc1d6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:04.754708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf8dc6-06ff-4c9f-873c-ddffc6fdac05 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.755070 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf8dc6-06ff-4c9f-873c-ddffc6fdac05 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.755289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf8dc6-06ff-4c9f-873c-ddffc6fdac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.792966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-17bf8dc6-06ff-4c9f-873c-ddffc6fdac05 None None] MFA Rules not processed for user `ff204b48ac0d4eccb05e339c6adeaefe`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:04.801473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-17bf8dc6-06ff-4c9f-873c-ddffc6fdac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:04.858866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb52dc-b6e9-463b-9cfb-48be4020d02e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:04.859622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb52dc-b6e9-463b-9cfb-48be4020d02e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:04.860013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb52dc-b6e9-463b-9cfb-48be4020d02e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:04.860487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb52dc-b6e9-463b-9cfb-48be4020d02e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:04.884041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:04.915582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1252/2511] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:04.922255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf8dc6-06ff-4c9f-873c-ddffc6fdac05 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:04.949461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb52dc-b6e9-463b-9cfb-48be4020d02e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:04.951081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb52dc-b6e9-463b-9cfb-48be4020d02e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:04.978463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1260/2512] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:05.108473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf8dc6-06ff-4c9f-873c-ddffc6fdac05 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:05.112966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad06f7-ed21-436a-bd0e-b746a59e24ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:05.113984 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad06f7-ed21-436a-bd0e-b746a59e24ba admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:05.114216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad06f7-ed21-436a-bd0e-b746a59e24ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:05.114594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad06f7-ed21-436a-bd0e-b746a59e24ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:05.115598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad06f7-ed21-436a-bd0e-b746a59e24ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:05.117757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad06f7-ed21-436a-bd0e-b746a59e24ba admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:05.155446 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1253/2513] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:04 2022] POST /identity/v3/projects => generated 373 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:05.159123 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb52dc-b6e9-463b-9cfb-48be4020d02e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:05.171401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb52dc-b6e9-463b-9cfb-48be4020d02e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:05.204462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:05.251565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258c496-ff22-470c-825b-d090fde79626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:05.252909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258c496-ff22-470c-825b-d090fde79626 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:05.253453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258c496-ff22-470c-825b-d090fde79626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:05.254209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258c496-ff22-470c-825b-d090fde79626 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:05.254818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258c496-ff22-470c-825b-d090fde79626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:05.256896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258c496-ff22-470c-825b-d090fde79626 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:05.297896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1261/2514] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:05 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:05.302481 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad06f7-ed21-436a-bd0e-b746a59e24ba admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:05.318422 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad06f7-ed21-436a-bd0e-b746a59e24ba admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:05.324086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c678e9-1f0c-4b40-9530-347f25d7e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:05.324948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c678e9-1f0c-4b40-9530-347f25d7e6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:05.325238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c678e9-1f0c-4b40-9530-347f25d7e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:05.325518 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c678e9-1f0c-4b40-9530-347f25d7e6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:05.326257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c678e9-1f0c-4b40-9530-347f25d7e6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:05.328105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c678e9-1f0c-4b40-9530-347f25d7e6bd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:05.353064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1254/2515] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:05.354835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258c496-ff22-470c-825b-d090fde79626 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:05.370237 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258c496-ff22-470c-825b-d090fde79626 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:05.378580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321a7fe-1c62-474e-b04f-c98b73dd5220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:05.379573 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321a7fe-1c62-474e-b04f-c98b73dd5220 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:05.384569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321a7fe-1c62-474e-b04f-c98b73dd5220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:05.384569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321a7fe-1c62-474e-b04f-c98b73dd5220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:05.384569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321a7fe-1c62-474e-b04f-c98b73dd5220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:05.387761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321a7fe-1c62-474e-b04f-c98b73dd5220 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:05.404314 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1262/2516] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:05.405378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c678e9-1f0c-4b40-9530-347f25d7e6bd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:05.417588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c678e9-1f0c-4b40-9530-347f25d7e6bd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:05.423793 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a747-716a-4c8b-9213-3d84cb1faff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:05.425411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a747-716a-4c8b-9213-3d84cb1faff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:05.425734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a747-716a-4c8b-9213-3d84cb1faff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:05.426073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a747-716a-4c8b-9213-3d84cb1faff9 admin admin] PATH_INFO: `/v3/projects/a06ae2f9cbf64363a685f857556dc268/users/6bec043243734700ae8a9c5a5ed27a4d/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:05.449836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1a747-716a-4c8b-9213-3d84cb1faff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06ae2f9cbf64363a685f857556dc268, user_id=6bec043243734700ae8a9c5a5ed27a4d, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:05.455036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1a747-716a-4c8b-9213-3d84cb1faff9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:05.477049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1255/2517] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:05 2022] PUT /identity/v3/projects/a06ae2f9cbf64363a685f857556dc268/users/6bec043243734700ae8a9c5a5ed27a4d/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:05.497589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67cf3-3ed1-4489-abbd-abbd1478e606 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:05.543996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67cf3-3ed1-4489-abbd-abbd1478e606 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:05.545454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67cf3-3ed1-4489-abbd-abbd1478e606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:05.598662 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cae67cf3-3ed1-4489-abbd-abbd1478e606 None None] MFA Rules not processed for user `6bec043243734700ae8a9c5a5ed27a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:05.600048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cae67cf3-3ed1-4489-abbd-abbd1478e606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:05.704385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1263/2518] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:05.708684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a747-716a-4c8b-9213-3d84cb1faff9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:05.824605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a747-716a-4c8b-9213-3d84cb1faff9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:05.844331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a60a7-6d68-4206-9b4d-6ee10e379e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:05.845185 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a60a7-6d68-4206-9b4d-6ee10e379e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:05.845569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a60a7-6d68-4206-9b4d-6ee10e379e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:05.846077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a60a7-6d68-4206-9b4d-6ee10e379e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:05.847100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a60a7-6d68-4206-9b4d-6ee10e379e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:05.849150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a60a7-6d68-4206-9b4d-6ee10e379e35 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:05.920287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1256/2519] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:05 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:11:05.923807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae67cf3-3ed1-4489-abbd-abbd1478e606 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:05.941096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae67cf3-3ed1-4489-abbd-abbd1478e606 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.009652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a12d0-955a-4a3a-ad86-cf6cac08a66c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.010176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a12d0-955a-4a3a-ad86-cf6cac08a66c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.010642 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a12d0-955a-4a3a-ad86-cf6cac08a66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.050282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e63fe6-67ed-4606-887f-621d4f8e33d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.051420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63fe6-67ed-4606-887f-621d4f8e33d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.052041 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63fe6-67ed-4606-887f-621d4f8e33d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.052516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63fe6-67ed-4606-887f-621d4f8e33d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.053409 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e63fe6-67ed-4606-887f-621d4f8e33d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.057107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e63fe6-67ed-4606-887f-621d4f8e33d7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.059970 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-007a12d0-955a-4a3a-ad86-cf6cac08a66c None None] MFA Rules not processed for user `062d003aac4e4521bc6c5bd8123d95d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:06.061372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-007a12d0-955a-4a3a-ad86-cf6cac08a66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:06.076031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:06.100873 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1264/2520] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:05 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:06.103204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e63fe6-67ed-4606-887f-621d4f8e33d7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.119090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e63fe6-67ed-4606-887f-621d4f8e33d7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.146738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e003b2-e174-4117-82dc-89504ae5ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.149048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e003b2-e174-4117-82dc-89504ae5ae32 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.149899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e003b2-e174-4117-82dc-89504ae5ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.150503 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e003b2-e174-4117-82dc-89504ae5ae32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.152841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e003b2-e174-4117-82dc-89504ae5ae32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.156039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e003b2-e174-4117-82dc-89504ae5ae32 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.180882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1265/2521] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:06.188763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e003b2-e174-4117-82dc-89504ae5ae32 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.196938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1257/2522] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:06.202450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a12d0-955a-4a3a-ad86-cf6cac08a66c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.212080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e003b2-e174-4117-82dc-89504ae5ae32 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.227317 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e793-881d-43ed-aa94-7cc1051aed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.228805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e793-881d-43ed-aa94-7cc1051aed69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.229088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e793-881d-43ed-aa94-7cc1051aed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.229366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261e793-881d-43ed-aa94-7cc1051aed69 admin admin] PATH_INFO: `/v3/projects/d673597c2e184f62b8963b529cc358df/users/cee2d2b314b94f24b5cbf07eb57eb831/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.230875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a12d0-955a-4a3a-ad86-cf6cac08a66c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.234288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:06.247242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cdf95-3894-4106-b485-59096167020c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.247958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cdf95-3894-4106-b485-59096167020c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.248224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cdf95-3894-4106-b485-59096167020c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.248647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cdf95-3894-4106-b485-59096167020c admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.249348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cdf95-3894-4106-b485-59096167020c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.253189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cdf95-3894-4106-b485-59096167020c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.273220 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261e793-881d-43ed-aa94-7cc1051aed69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d673597c2e184f62b8963b529cc358df, user_id=cee2d2b314b94f24b5cbf07eb57eb831, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.276213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261e793-881d-43ed-aa94-7cc1051aed69 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.311955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1266/2523] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:06 2022] PUT /identity/v3/projects/d673597c2e184f62b8963b529cc358df/users/cee2d2b314b94f24b5cbf07eb57eb831/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:06.313412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e793-881d-43ed-aa94-7cc1051aed69 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.314385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1258/2524] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:06 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:11:06.320408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cdf95-3894-4106-b485-59096167020c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.461102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261e793-881d-43ed-aa94-7cc1051aed69 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.462646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cdf95-3894-4106-b485-59096167020c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.468412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a0b9c-d88e-4781-b344-381cb8579d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.468412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a0b9c-d88e-4781-b344-381cb8579d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.468412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a0b9c-d88e-4781-b344-381cb8579d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.468412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a0b9c-d88e-4781-b344-381cb8579d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.472785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a939c-b7ed-4189-a900-b7acaeca7bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.473004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a0b9c-d88e-4781-b344-381cb8579d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.473841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a939c-b7ed-4189-a900-b7acaeca7bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.474298 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a939c-b7ed-4189-a900-b7acaeca7bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.474676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a0b9c-d88e-4781-b344-381cb8579d44 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.474897 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3a939c-b7ed-4189-a900-b7acaeca7bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.475917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a939c-b7ed-4189-a900-b7acaeca7bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.478862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3a939c-b7ed-4189-a900-b7acaeca7bf3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.488578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1267/2525] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:06.493113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a0b9c-d88e-4781-b344-381cb8579d44 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.510201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a0b9c-d88e-4781-b344-381cb8579d44 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.513980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1259/2526] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:06 2022] POST /identity/v3/users => generated 308 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:06.518713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a939c-b7ed-4189-a900-b7acaeca7bf3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.519751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a52c88-05f5-4d7a-9c0f-8878c155ce17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.520695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a52c88-05f5-4d7a-9c0f-8878c155ce17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.520988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a52c88-05f5-4d7a-9c0f-8878c155ce17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.521287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a52c88-05f5-4d7a-9c0f-8878c155ce17 admin admin] PATH_INFO: `/v3/projects/d673597c2e184f62b8963b529cc358df/users/cee2d2b314b94f24b5cbf07eb57eb831/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.523315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a52c88-05f5-4d7a-9c0f-8878c155ce17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d673597c2e184f62b8963b529cc358df, user_id=cee2d2b314b94f24b5cbf07eb57eb831, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.530289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a52c88-05f5-4d7a-9c0f-8878c155ce17 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.530510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3a939c-b7ed-4189-a900-b7acaeca7bf3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.537847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b8cc7-4e4f-4909-b3b6-4ce7b465f1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.538340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b8cc7-4e4f-4909-b3b6-4ce7b465f1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.538625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b8cc7-4e4f-4909-b3b6-4ce7b465f1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.539140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b8cc7-4e4f-4909-b3b6-4ce7b465f1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.539880 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b8cc7-4e4f-4909-b3b6-4ce7b465f1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.541548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b8cc7-4e4f-4909-b3b6-4ce7b465f1d2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.544196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1268/2527] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:06 2022] PUT /identity/v3/projects/d673597c2e184f62b8963b529cc358df/users/cee2d2b314b94f24b5cbf07eb57eb831/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:06.553548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5083e-d62e-4b9c-9d50-fbea497740c4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.553849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5083e-d62e-4b9c-9d50-fbea497740c4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.554140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5083e-d62e-4b9c-9d50-fbea497740c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.567603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1260/2528] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:06.571644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b8cc7-4e4f-4909-b3b6-4ce7b465f1d2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.587064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b8cc7-4e4f-4909-b3b6-4ce7b465f1d2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.620431 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-27d5083e-d62e-4b9c-9d50-fbea497740c4 None None] MFA Rules not processed for user `cee2d2b314b94f24b5cbf07eb57eb831`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:06.622528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-27d5083e-d62e-4b9c-9d50-fbea497740c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:06.694470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f41d-f1a5-47db-9088-4c1b771fd2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.695721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f41d-f1a5-47db-9088-4c1b771fd2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.696012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f41d-f1a5-47db-9088-4c1b771fd2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.696324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f41d-f1a5-47db-9088-4c1b771fd2f1 admin admin] PATH_INFO: `/v3/projects/fb5691d836f24584a8484b41bdf8726a/users/1a22330a10c54e829188118bf630f517/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.714072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:06.719462 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39f41d-f1a5-47db-9088-4c1b771fd2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5691d836f24584a8484b41bdf8726a, user_id=1a22330a10c54e829188118bf630f517, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.724345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39f41d-f1a5-47db-9088-4c1b771fd2f1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.732349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1269/2529] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:06.742467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5083e-d62e-4b9c-9d50-fbea497740c4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.744200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-27d5083e-d62e-4b9c-9d50-fbea497740c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:06.745050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1261/2530] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:06 2022] PUT /identity/v3/projects/fb5691d836f24584a8484b41bdf8726a/users/1a22330a10c54e829188118bf630f517/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:06.749126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f41d-f1a5-47db-9088-4c1b771fd2f1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.825578 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5083e-d62e-4b9c-9d50-fbea497740c4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.829945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:06.843046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f41d-f1a5-47db-9088-4c1b771fd2f1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.850904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595978b-0a92-4c2b-9775-91954de1b63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.851598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595978b-0a92-4c2b-9775-91954de1b63c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.851883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595978b-0a92-4c2b-9775-91954de1b63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.852335 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595978b-0a92-4c2b-9775-91954de1b63c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.853089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595978b-0a92-4c2b-9775-91954de1b63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.854746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595978b-0a92-4c2b-9775-91954de1b63c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.872640 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1262/2531] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:06.873626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595978b-0a92-4c2b-9775-91954de1b63c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:06.888658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595978b-0a92-4c2b-9775-91954de1b63c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:06.898597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.898862 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae0c8b-c694-45d2-b488-8e292c6f8b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:06.899302 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.899565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.899847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.900005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0c8b-c694-45d2-b488-8e292c6f8b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:06.900447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0c8b-c694-45d2-b488-8e292c6f8b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:06.900797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0c8b-c694-45d2-b488-8e292c6f8b44 admin admin] PATH_INFO: `/v3/projects/fb5691d836f24584a8484b41bdf8726a/users/1a22330a10c54e829188118bf630f517/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:06.902979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae0c8b-c694-45d2-b488-8e292c6f8b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb5691d836f24584a8484b41bdf8726a, user_id=1a22330a10c54e829188118bf630f517, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.906277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae0c8b-c694-45d2-b488-8e292c6f8b44 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.945141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1263/2532] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:06 2022] PUT /identity/v3/projects/fb5691d836f24584a8484b41bdf8726a/users/1a22330a10c54e829188118bf630f517/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:06.982964 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:06.984846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:06.996505 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1270/2533] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:11:07.806594 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05772fa-020b-4336-8b8a-22353f4aaedf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:07.807169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05772fa-020b-4336-8b8a-22353f4aaedf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:07.807169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05772fa-020b-4336-8b8a-22353f4aaedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:07.836798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c05772fa-020b-4336-8b8a-22353f4aaedf None None] MFA Rules not processed for user `1a22330a10c54e829188118bf630f517`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:07.838045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c05772fa-020b-4336-8b8a-22353f4aaedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:07.937002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1264/2534] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:08.754925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:08.757058 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:08.775747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059fbcf-259b-4607-8bbb-77742dd72acf service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:08.843116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a5767-4bed-4729-9287-8473c1a40a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:08.843891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a5767-4bed-4729-9287-8473c1a40a4b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:08.844288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a5767-4bed-4729-9287-8473c1a40a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:08.844646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a5767-4bed-4729-9287-8473c1a40a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:08.927794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a5767-4bed-4729-9287-8473c1a40a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:08.929442 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a5767-4bed-4729-9287-8473c1a40a4b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:08.942808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1271/2535] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:09.329014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05772fa-020b-4336-8b8a-22353f4aaedf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:09.417939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:09.424491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05772fa-020b-4336-8b8a-22353f4aaedf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:09.431265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:09.432570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:09.433033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:09.433480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] PATH_INFO: `/v3/users/062d003aac4e4521bc6c5bd8123d95d9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:09.434713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=062d003aac4e4521bc6c5bd8123d95d9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:09.436893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:09.514719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062d003aac4e4521bc6c5bd8123d95d9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:09.535118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062d003aac4e4521bc6c5bd8123d95d9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:09.535458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062d003aac4e4521bc6c5bd8123d95d9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:09.545104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062d003aac4e4521bc6c5bd8123d95d9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:09.545832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062d003aac4e4521bc6c5bd8123d95d9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:09.568202 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1265/2536] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:09 2022] DELETE /identity/v3/users/062d003aac4e4521bc6c5bd8123d95d9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:09.794222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a5767-4bed-4729-9287-8473c1a40a4b service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:09.805530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a5767-4bed-4729-9287-8473c1a40a4b service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:09.814396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:09.816270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:09.824959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:09.905737 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:09.907070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:09.910241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:09.910241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:09.918050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:09.946982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d4e9d-e795-4d0d-8081-b7513d5541bc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:09.964726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:09.968843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:09.969388 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:09.969672 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] PATH_INFO: `/v3/projects/b99ee7dfe9dc4e98a38d3463bc0464b4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:09.973034 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99ee7dfe9dc4e98a38d3463bc0464b4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:09.975979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:10.058923 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:10.060485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:10.066746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99ee7dfe9dc4e98a38d3463bc0464b4', 'request_id': 'req-4cd86c0b-53b5-49ee-aea2-f63143125b79'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.074720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99ee7dfe9dc4e98a38d3463bc0464b4', 'request_id': 'req-4cd86c0b-53b5-49ee-aea2-f63143125b79'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.086491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99ee7dfe9dc4e98a38d3463bc0464b4', 'request_id': 'req-4cd86c0b-53b5-49ee-aea2-f63143125b79'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.090501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1272/2537] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:10.115740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] The token cache is being invalidate because project b99ee7dfe9dc4e98a38d3463bc0464b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:10.116137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.117125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.119530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1266/2538] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:09 2022] DELETE /identity/v3/projects/b99ee7dfe9dc4e98a38d3463bc0464b4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:10.124029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:10.141690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b0b52-cdd3-4b37-935f-fd1479e1d79d service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:10.231392 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:10.232691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:10.233052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:10.233539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] PATH_INFO: `/v3/users/1a22330a10c54e829188118bf630f517` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:10.234878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a22330a10c54e829188118bf630f517)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:10.237851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:10.310012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a22330a10c54e829188118bf630f517'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.327516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a22330a10c54e829188118bf630f517'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.328588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a22330a10c54e829188118bf630f517'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.329192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a22330a10c54e829188118bf630f517'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.333339 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:10.341804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f1d9a9b-31f6-43e5-bea4-d98bf0dfd393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a22330a10c54e829188118bf630f517'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.364554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1273/2539] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:10 2022] DELETE /identity/v3/users/1a22330a10c54e829188118bf630f517 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:10.586911 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:10.588506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:10.608788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd86c0b-53b5-49ee-aea2-f63143125b79 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:10.613864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:10.693968 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:10.695029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:10.695411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:10.695795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] PATH_INFO: `/v3/projects/fb5691d836f24584a8484b41bdf8726a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:10.697193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb5691d836f24584a8484b41bdf8726a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:10.699313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:10.756588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb5691d836f24584a8484b41bdf8726a', 'request_id': 'req-8a785561-f89b-4d9f-af1a-4386ffe4118f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.763939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb5691d836f24584a8484b41bdf8726a', 'request_id': 'req-8a785561-f89b-4d9f-af1a-4386ffe4118f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.773240 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb5691d836f24584a8484b41bdf8726a', 'request_id': 'req-8a785561-f89b-4d9f-af1a-4386ffe4118f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.790337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] The token cache is being invalidate because project fb5691d836f24584a8484b41bdf8726a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:10.790637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.791340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a785561-f89b-4d9f-af1a-4386ffe4118f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:10.797026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1267/2540] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:10 2022] DELETE /identity/v3/projects/fb5691d836f24584a8484b41bdf8726a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:10.798374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1063e8-05ad-49a4-923d-25065d8a6aac None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:10.799559 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1063e8-05ad-49a4-923d-25065d8a6aac None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:10.799812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1063e8-05ad-49a4-923d-25065d8a6aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:10.826533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aa1063e8-05ad-49a4-923d-25065d8a6aac None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:10.829632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa1063e8-05ad-49a4-923d-25065d8a6aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:10.901264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1274/2541] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:10.909739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefc0f2-980e-43bf-b3fb-79ad7dca113c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:10.910027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefc0f2-980e-43bf-b3fb-79ad7dca113c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:10.910287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefc0f2-980e-43bf-b3fb-79ad7dca113c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:10.929121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3aefc0f2-980e-43bf-b3fb-79ad7dca113c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:10.930470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3aefc0f2-980e-43bf-b3fb-79ad7dca113c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:10.938753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1268/2542] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:10.943556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1063e8-05ad-49a4-923d-25065d8a6aac None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:10.957036 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1063e8-05ad-49a4-923d-25065d8a6aac None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:10.963024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:10.963794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:10.964152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:10.964550 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:10.965289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:10.965602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:10.967221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:10.985108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefc0f2-980e-43bf-b3fb-79ad7dca113c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:10.990941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1275/2543] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:10.994908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefc0f2-980e-43bf-b3fb-79ad7dca113c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:10.999646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0013943-3bb9-45c7-bd3a-1031d66952b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.000770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0013943-3bb9-45c7-bd3a-1031d66952b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.000770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0013943-3bb9-45c7-bd3a-1031d66952b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.000998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0013943-3bb9-45c7-bd3a-1031d66952b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.002283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0013943-3bb9-45c7-bd3a-1031d66952b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.004094 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0013943-3bb9-45c7-bd3a-1031d66952b4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.039764 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1269/2544] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:10 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:11:11.043763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.055251 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ae4f-33b8-4aa3-afe0-0c96fa5fb397 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.125871 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe47d4-3cce-4a7d-8ceb-4010bdcf2f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.126552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe47d4-3cce-4a7d-8ceb-4010bdcf2f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.126843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe47d4-3cce-4a7d-8ceb-4010bdcf2f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.127188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe47d4-3cce-4a7d-8ceb-4010bdcf2f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.128049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe47d4-3cce-4a7d-8ceb-4010bdcf2f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.130078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe47d4-3cce-4a7d-8ceb-4010bdcf2f7e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.158916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1276/2545] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:11 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:11.162784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0013943-3bb9-45c7-bd3a-1031d66952b4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.176072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0013943-3bb9-45c7-bd3a-1031d66952b4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.183558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68e6ea-0780-4267-a537-985d57dbc80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.184281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e6ea-0780-4267-a537-985d57dbc80c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.184584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e6ea-0780-4267-a537-985d57dbc80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.184852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68e6ea-0780-4267-a537-985d57dbc80c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.185598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68e6ea-0780-4267-a537-985d57dbc80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.187217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68e6ea-0780-4267-a537-985d57dbc80c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.199362 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1270/2546] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:11.204436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe47d4-3cce-4a7d-8ceb-4010bdcf2f7e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.216284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe47d4-3cce-4a7d-8ceb-4010bdcf2f7e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.223693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35795a-ff43-4774-bd74-f75a13478961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.224487 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35795a-ff43-4774-bd74-f75a13478961 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.224784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35795a-ff43-4774-bd74-f75a13478961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.225085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35795a-ff43-4774-bd74-f75a13478961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.225809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35795a-ff43-4774-bd74-f75a13478961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.227547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35795a-ff43-4774-bd74-f75a13478961 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.242930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1277/2547] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:11.244038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68e6ea-0780-4267-a537-985d57dbc80c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.255485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68e6ea-0780-4267-a537-985d57dbc80c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.260485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2ce9a-0c67-40d7-90ae-2a22d6ba26e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.261796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2ce9a-0c67-40d7-90ae-2a22d6ba26e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.261796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2ce9a-0c67-40d7-90ae-2a22d6ba26e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.262167 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2ce9a-0c67-40d7-90ae-2a22d6ba26e8 admin admin] PATH_INFO: `/v3/projects/fc39b1a7a90441ee9085dc2642369279/users/85ea651be48e458aa83a590268a790cd/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.284729 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2ce9a-0c67-40d7-90ae-2a22d6ba26e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc39b1a7a90441ee9085dc2642369279, user_id=85ea651be48e458aa83a590268a790cd, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.287548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2ce9a-0c67-40d7-90ae-2a22d6ba26e8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.317245 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1271/2548] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:11 2022] PUT /identity/v3/projects/fc39b1a7a90441ee9085dc2642369279/users/85ea651be48e458aa83a590268a790cd/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:11.319101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae0079-5de8-4896-8c56-e8f51b0d5f5c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.319538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae0079-5de8-4896-8c56-e8f51b0d5f5c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.319924 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae0079-5de8-4896-8c56-e8f51b0d5f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.355170 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9eae0079-5de8-4896-8c56-e8f51b0d5f5c None None] MFA Rules not processed for user `85ea651be48e458aa83a590268a790cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:11.356627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9eae0079-5de8-4896-8c56-e8f51b0d5f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:11.413992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:11.458023 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1278/2549] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:11.462399 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2ce9a-0c67-40d7-90ae-2a22d6ba26e8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.524639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:11.573379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2ce9a-0c67-40d7-90ae-2a22d6ba26e8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.581149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb1632-4946-49ee-9c01-be338b800474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.582085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1632-4946-49ee-9c01-be338b800474 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.582783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1632-4946-49ee-9c01-be338b800474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.583148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1632-4946-49ee-9c01-be338b800474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.584107 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb1632-4946-49ee-9c01-be338b800474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.586570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb1632-4946-49ee-9c01-be338b800474 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.629067 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae0079-5de8-4896-8c56-e8f51b0d5f5c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.629575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1272/2550] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:11 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:11.652746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae0079-5de8-4896-8c56-e8f51b0d5f5c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.736102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245d769-a517-44eb-9e19-abcaa24033ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.736888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d769-a517-44eb-9e19-abcaa24033ec admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.737222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d769-a517-44eb-9e19-abcaa24033ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.739590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d769-a517-44eb-9e19-abcaa24033ec admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.740348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245d769-a517-44eb-9e19-abcaa24033ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.742327 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245d769-a517-44eb-9e19-abcaa24033ec admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.773316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1279/2551] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:11 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:11.774967 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb1632-4946-49ee-9c01-be338b800474 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.788940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb1632-4946-49ee-9c01-be338b800474 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.796801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f38c81-4f87-4bd5-82c2-04aba83027b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.797709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f38c81-4f87-4bd5-82c2-04aba83027b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.798196 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f38c81-4f87-4bd5-82c2-04aba83027b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.798477 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f38c81-4f87-4bd5-82c2-04aba83027b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.799809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f38c81-4f87-4bd5-82c2-04aba83027b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.801846 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f38c81-4f87-4bd5-82c2-04aba83027b6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.814274 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1273/2552] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:11.818982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245d769-a517-44eb-9e19-abcaa24033ec admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.832399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245d769-a517-44eb-9e19-abcaa24033ec admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.837837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3a1b-a1ad-4a0a-9336-c6003fb28a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.838796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3a1b-a1ad-4a0a-9336-c6003fb28a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.839088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3a1b-a1ad-4a0a-9336-c6003fb28a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.839369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3a1b-a1ad-4a0a-9336-c6003fb28a26 admin admin] PATH_INFO: `/v3/projects/36bfc5ff5492473a9ff8fab02efff02e/users/56cb1caab843481fa85e340cf9fa1b57/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.857208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b3a1b-a1ad-4a0a-9336-c6003fb28a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36bfc5ff5492473a9ff8fab02efff02e, user_id=56cb1caab843481fa85e340cf9fa1b57, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.860104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b3a1b-a1ad-4a0a-9336-c6003fb28a26 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.883460 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1280/2553] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:11 2022] PUT /identity/v3/projects/36bfc5ff5492473a9ff8fab02efff02e/users/56cb1caab843481fa85e340cf9fa1b57/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:11.885644 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f38c81-4f87-4bd5-82c2-04aba83027b6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:11.896899 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f38c81-4f87-4bd5-82c2-04aba83027b6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:11.965074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb5bd4-2253-4e64-ae43-82dcef416986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:11.965905 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb5bd4-2253-4e64-ae43-82dcef416986 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:11.966203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb5bd4-2253-4e64-ae43-82dcef416986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:11.966532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb5bd4-2253-4e64-ae43-82dcef416986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:11.967300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb5bd4-2253-4e64-ae43-82dcef416986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:11.969198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb5bd4-2253-4e64-ae43-82dcef416986 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:11.981740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1274/2554] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:11.989154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3a1b-a1ad-4a0a-9336-c6003fb28a26 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:12.002438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3a1b-a1ad-4a0a-9336-c6003fb28a26 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:12.009155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:12.010089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:12.010374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:12.010656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] PATH_INFO: `/v3/projects/36bfc5ff5492473a9ff8fab02efff02e/users/56cb1caab843481fa85e340cf9fa1b57/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:12.012884 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36bfc5ff5492473a9ff8fab02efff02e, user_id=56cb1caab843481fa85e340cf9fa1b57, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:12.015665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:12.032201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1281/2555] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:11 2022] PUT /identity/v3/projects/36bfc5ff5492473a9ff8fab02efff02e/users/56cb1caab843481fa85e340cf9fa1b57/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:12.039716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd6169-9557-47ca-8530-1c1afc7b73af None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:12.040037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd6169-9557-47ca-8530-1c1afc7b73af None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:12.040346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd6169-9557-47ca-8530-1c1afc7b73af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:12.072082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-07cd6169-9557-47ca-8530-1c1afc7b73af None None] MFA Rules not processed for user `56cb1caab843481fa85e340cf9fa1b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:12.073387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-07cd6169-9557-47ca-8530-1c1afc7b73af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:12.181936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1275/2556] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:12.193386 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:12.195200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:12.266014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c899635-32a7-4ef4-9648-06f2ed56ead3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:12.271465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9bb10-96e4-4287-b9ea-356cdfed2dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:12.272208 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bb10-96e4-4287-b9ea-356cdfed2dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:12.272506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bb10-96e4-4287-b9ea-356cdfed2dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:12.272779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bb10-96e4-4287-b9ea-356cdfed2dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:12.287198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9bb10-96e4-4287-b9ea-356cdfed2dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:12.290035 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9bb10-96e4-4287-b9ea-356cdfed2dd9 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:12.311017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1282/2557] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:12 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:12.823060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a631f-be6d-4cd7-a964-f25f5ce20889 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:12.823591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a631f-be6d-4cd7-a964-f25f5ce20889 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:12.824111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a631f-be6d-4cd7-a964-f25f5ce20889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:12.855239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f08a631f-be6d-4cd7-a964-f25f5ce20889 None None] MFA Rules not processed for user `c8915c74c4d24d2cb1cded5aa7927120`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:12.856548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f08a631f-be6d-4cd7-a964-f25f5ce20889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:12.890648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:12.928027 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1276/2558] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:13.840869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9bb10-96e4-4287-b9ea-356cdfed2dd9 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:13.884204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:13.912570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9bb10-96e4-4287-b9ea-356cdfed2dd9 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:13.917215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:13.918092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:13.918376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:13.918652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] PATH_INFO: `/v3/users/85ea651be48e458aa83a590268a790cd` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:13.919885 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ea651be48e458aa83a590268a790cd)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:13.921867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:13.973743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ea651be48e458aa83a590268a790cd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:13.986045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ea651be48e458aa83a590268a790cd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:13.987286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ea651be48e458aa83a590268a790cd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:13.988127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ea651be48e458aa83a590268a790cd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:13.994993 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ea651be48e458aa83a590268a790cd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.009929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1283/2559] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:13 2022] DELETE /identity/v3/users/85ea651be48e458aa83a590268a790cd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:14.014347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6aab-3700-4a61-bfb1-7fb63cdef02b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:14.014840 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6aab-3700-4a61-bfb1-7fb63cdef02b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:14.014956 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6aab-3700-4a61-bfb1-7fb63cdef02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:14.033885 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-61ae6aab-3700-4a61-bfb1-7fb63cdef02b None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:14.035180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-61ae6aab-3700-4a61-bfb1-7fb63cdef02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:14.097568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1277/2560] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:14.107066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:14.108509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:14.188865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae6aab-3700-4a61-bfb1-7fb63cdef02b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:14.189204 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ae8e-3613-4629-92f0-6eb6dbcb175e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:14.190746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-61ae6aab-3700-4a61-bfb1-7fb63cdef02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:14.193393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e534547-40ba-44d5-b2ea-103f62d67f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:14.194057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e534547-40ba-44d5-b2ea-103f62d67f5e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:14.194356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e534547-40ba-44d5-b2ea-103f62d67f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:14.194689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e534547-40ba-44d5-b2ea-103f62d67f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:14.204451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae6aab-3700-4a61-bfb1-7fb63cdef02b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:14.206554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e534547-40ba-44d5-b2ea-103f62d67f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:14.208111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e534547-40ba-44d5-b2ea-103f62d67f5e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:14.210677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:14.211653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:14.211947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:14.212395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] PATH_INFO: `/v3/users/ff204b48ac0d4eccb05e339c6adeaefe` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:14.213621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff204b48ac0d4eccb05e339c6adeaefe)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:14.215576 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:14.218575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1284/2561] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:14.269958 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff204b48ac0d4eccb05e339c6adeaefe'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.284131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff204b48ac0d4eccb05e339c6adeaefe'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.284922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff204b48ac0d4eccb05e339c6adeaefe'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.292617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff204b48ac0d4eccb05e339c6adeaefe'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.293338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff204b48ac0d4eccb05e339c6adeaefe'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.308104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1278/2562] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:14 2022] DELETE /identity/v3/users/ff204b48ac0d4eccb05e339c6adeaefe => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:14.312716 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1761b7f-8839-4945-ab5f-3b0f83525513 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:14.313038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1761b7f-8839-4945-ab5f-3b0f83525513 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:14.313328 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1761b7f-8839-4945-ab5f-3b0f83525513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:14.332718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d1761b7f-8839-4945-ab5f-3b0f83525513 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:14.333948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d1761b7f-8839-4945-ab5f-3b0f83525513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:14.396050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:14.407941 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1285/2563] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:14.419866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:14.421376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:14.486768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:14.498260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b77465-fd9d-463c-9636-97516464474c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:14.511682 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d4dc-ea86-4984-803d-592c6fc02988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:14.512980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d4dc-ea86-4984-803d-592c6fc02988 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:14.513621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d4dc-ea86-4984-803d-592c6fc02988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:14.513883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d4dc-ea86-4984-803d-592c6fc02988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:14.518475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1761b7f-8839-4945-ab5f-3b0f83525513 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:14.519886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d1761b7f-8839-4945-ab5f-3b0f83525513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:14.534430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1761b7f-8839-4945-ab5f-3b0f83525513 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:14.538367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261d4dc-ea86-4984-803d-592c6fc02988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:14.540717 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261d4dc-ea86-4984-803d-592c6fc02988 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:14.556786 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:14.557752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:14.558043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:14.558324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] PATH_INFO: `/v3/projects/fc39b1a7a90441ee9085dc2642369279` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:14.559624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc39b1a7a90441ee9085dc2642369279)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:14.561731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:14.564545 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1279/2564] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:11:14.699904 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc39b1a7a90441ee9085dc2642369279', 'request_id': 'req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.713070 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc39b1a7a90441ee9085dc2642369279', 'request_id': 'req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.725380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc39b1a7a90441ee9085dc2642369279', 'request_id': 'req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.738379 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] The token cache is being invalidate because project fc39b1a7a90441ee9085dc2642369279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:14.738674 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.739387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a0b12271-fd8b-4b7b-9fdc-aa199aeed3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.743979 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1286/2565] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:14 2022] DELETE /identity/v3/projects/fc39b1a7a90441ee9085dc2642369279 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:14.745241 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d4dc-ea86-4984-803d-592c6fc02988 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:14.761312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d4dc-ea86-4984-803d-592c6fc02988 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:14.849577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:14.850468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:14.850758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:14.851041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] PATH_INFO: `/v3/users/56cb1caab843481fa85e340cf9fa1b57` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:14.852535 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56cb1caab843481fa85e340cf9fa1b57)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:14.854731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:14.924552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56cb1caab843481fa85e340cf9fa1b57'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.939045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56cb1caab843481fa85e340cf9fa1b57'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.940121 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56cb1caab843481fa85e340cf9fa1b57'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.948009 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56cb1caab843481fa85e340cf9fa1b57'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.948795 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56cb1caab843481fa85e340cf9fa1b57'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:14.968249 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1280/2566] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:14 2022] DELETE /identity/v3/users/56cb1caab843481fa85e340cf9fa1b57 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:14.987690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1da0c-4e3f-4d9c-95c3-fdecd89a65df None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:14.988903 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1da0c-4e3f-4d9c-95c3-fdecd89a65df None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:14.989394 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1da0c-4e3f-4d9c-95c3-fdecd89a65df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.010466 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-90b1da0c-4e3f-4d9c-95c3-fdecd89a65df None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:15.011803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90b1da0c-4e3f-4d9c-95c3-fdecd89a65df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:15.061443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:15.062940 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:15.123632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1287/2567] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:11:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:15.134039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1da0c-4e3f-4d9c-95c3-fdecd89a65df None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:15.137976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90b1da0c-4e3f-4d9c-95c3-fdecd89a65df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:15.175751 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0a072-c9e5-4fdd-b4db-cbdfbc082a55 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:15.183603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:15.184731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:15.185118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:15.185679 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] PATH_INFO: `/v3/projects/5f8b14b6ddb049b6ae4a02df41c1663a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.187239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8b14b6ddb049b6ae4a02df41c1663a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:15.189732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:15.203342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:15.246130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1da0c-4e3f-4d9c-95c3-fdecd89a65df None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:15.255173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8b14b6ddb049b6ae4a02df41c1663a', 'request_id': 'req-2ee2f9ce-7114-4136-9013-44fb2f0605ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.265534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8b14b6ddb049b6ae4a02df41c1663a', 'request_id': 'req-2ee2f9ce-7114-4136-9013-44fb2f0605ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.274781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8b14b6ddb049b6ae4a02df41c1663a', 'request_id': 'req-2ee2f9ce-7114-4136-9013-44fb2f0605ce'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.282171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:15.293248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] The token cache is being invalidate because project 5f8b14b6ddb049b6ae4a02df41c1663a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:15.293492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.294510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.301028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1281/2568] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:15 2022] DELETE /identity/v3/projects/5f8b14b6ddb049b6ae4a02df41c1663a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:15.301489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:15.305019 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:15.337792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:15.338480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:15.338760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:15.339032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.340940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:15.384396 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2f9ce-7114-4136-9013-44fb2f0605ce admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:15.389097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:15.389985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:15.390267 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:15.390821 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] PATH_INFO: `/v3/users/6bec043243734700ae8a9c5a5ed27a4d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.391759 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bec043243734700ae8a9c5a5ed27a4d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:15.394721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:15.412380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:15.414351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:15.426753 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1288/2569] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:11:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 30 12:11:15.443699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:15.445648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:15.460313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bec043243734700ae8a9c5a5ed27a4d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.470094 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46691a-c350-4d20-b508-8924413e8824 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:15.479059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bec043243734700ae8a9c5a5ed27a4d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.479947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bec043243734700ae8a9c5a5ed27a4d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.481412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:15.482914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:15.483209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:15.483491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] PATH_INFO: `/v3/projects/36bfc5ff5492473a9ff8fab02efff02e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.485010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] RBAC: Authorizing `identity:delete_project(project_id=36bfc5ff5492473a9ff8fab02efff02e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:15.487872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:15.490128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bec043243734700ae8a9c5a5ed27a4d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.491269 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-bad5faa2-5e7f-45d0-ad38-2f24fc55f474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bec043243734700ae8a9c5a5ed27a4d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.512092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1282/2570] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:15 2022] DELETE /identity/v3/users/6bec043243734700ae8a9c5a5ed27a4d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:15.560597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36bfc5ff5492473a9ff8fab02efff02e', 'request_id': 'req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.575306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36bfc5ff5492473a9ff8fab02efff02e', 'request_id': 'req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.585071 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36bfc5ff5492473a9ff8fab02efff02e', 'request_id': 'req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.601499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] The token cache is being invalidate because project 36bfc5ff5492473a9ff8fab02efff02e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:15.602045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.606442 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8deaa8c1-717a-4011-b2d1-1abdcfe6a17f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:15.616868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7a4d3-01e6-4a09-b2e4-aeb7694308da None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:15.617476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7a4d3-01e6-4a09-b2e4-aeb7694308da None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:15.621881 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7a4d3-01e6-4a09-b2e4-aeb7694308da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.626219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1289/2571] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:15 2022] DELETE /identity/v3/projects/36bfc5ff5492473a9ff8fab02efff02e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:15.663685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-daa7a4d3-01e6-4a09-b2e4-aeb7694308da None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:15.672522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-daa7a4d3-01e6-4a09-b2e4-aeb7694308da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:15.708242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:15.809796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82093-6c84-4c42-8295-deff307277c7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:15.810344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82093-6c84-4c42-8295-deff307277c7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:15.810781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb82093-6c84-4c42-8295-deff307277c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.815414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1283/2572] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:15.837754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-feb82093-6c84-4c42-8295-deff307277c7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:15.840845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-feb82093-6c84-4c42-8295-deff307277c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:15.858546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7a4d3-01e6-4a09-b2e4-aeb7694308da None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:15.870271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1290/2573] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:15.874442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7a4d3-01e6-4a09-b2e4-aeb7694308da None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:15.889135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:15.891147 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:15.895999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:15.897735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.899575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:15.900623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:15.906389 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:15.948421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1284/2574] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:15.949524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb82093-6c84-4c42-8295-deff307277c7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:15.959198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb82093-6c84-4c42-8295-deff307277c7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:15.973113 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7957-efac-443f-a704-34e533637273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:15.973812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7957-efac-443f-a704-34e533637273 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:15.976453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7957-efac-443f-a704-34e533637273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:15.976763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7957-efac-443f-a704-34e533637273 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:15.977498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec7957-efac-443f-a704-34e533637273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:15.979471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec7957-efac-443f-a704-34e533637273 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:16.044641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1291/2575] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:15 2022] POST /identity/v3/projects => generated 399 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:11:16.052608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.071174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88693cf-4452-461e-8e3f-f4de40dbd724 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:16.171719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fd30d-fba2-4c22-8400-e1b623153a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:16.172558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fd30d-fba2-4c22-8400-e1b623153a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.172926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fd30d-fba2-4c22-8400-e1b623153a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.173271 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fd30d-fba2-4c22-8400-e1b623153a18 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.174664 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fd30d-fba2-4c22-8400-e1b623153a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:16.176999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fd30d-fba2-4c22-8400-e1b623153a18 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:16.216614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1285/2576] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:16 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:16.221850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7957-efac-443f-a704-34e533637273 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.239192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7957-efac-443f-a704-34e533637273 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:16.247870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96887d9c-764f-4558-b37e-b40921b7e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:16.250209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887d9c-764f-4558-b37e-b40921b7e00a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.250756 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887d9c-764f-4558-b37e-b40921b7e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.252640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887d9c-764f-4558-b37e-b40921b7e00a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.253675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96887d9c-764f-4558-b37e-b40921b7e00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:16.256051 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96887d9c-764f-4558-b37e-b40921b7e00a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:16.286891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1292/2577] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:16.290358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fd30d-fba2-4c22-8400-e1b623153a18 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.310925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fd30d-fba2-4c22-8400-e1b623153a18 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:16.316199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:16.316861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.317233 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.317427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.318257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:16.319917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:16.334796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1286/2578] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:16.339744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96887d9c-764f-4558-b37e-b40921b7e00a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.357901 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96887d9c-764f-4558-b37e-b40921b7e00a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:16.368768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cebd6-3bd8-4542-a7e2-8c8f5679b722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:16.369719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cebd6-3bd8-4542-a7e2-8c8f5679b722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.370148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cebd6-3bd8-4542-a7e2-8c8f5679b722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.370591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cebd6-3bd8-4542-a7e2-8c8f5679b722 admin admin] PATH_INFO: `/v3/projects/91cf9917f195476c9b9561c94c31255d/users/05e17fe44239485582511be5fc02e31c/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.397588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:16.417055 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cebd6-3bd8-4542-a7e2-8c8f5679b722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cf9917f195476c9b9561c94c31255d, user_id=05e17fe44239485582511be5fc02e31c, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:16.421034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cebd6-3bd8-4542-a7e2-8c8f5679b722 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:16.467019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1293/2579] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:16 2022] PUT /identity/v3/projects/91cf9917f195476c9b9561c94c31255d/users/05e17fe44239485582511be5fc02e31c/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:16.471146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.473747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e597-12e7-48c4-8944-4797506d2175 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.474110 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e597-12e7-48c4-8944-4797506d2175 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.474424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e597-12e7-48c4-8944-4797506d2175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.475835 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:16.495131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876d96f-7495-4d48-81c1-67497fd2342c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:16.524307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9aa4e597-12e7-48c4-8944-4797506d2175 None None] MFA Rules not processed for user `05e17fe44239485582511be5fc02e31c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:16.525678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9aa4e597-12e7-48c4-8944-4797506d2175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:16.659828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:16.661096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.661427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.661763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] PATH_INFO: `/v3/projects/a06ae2f9cbf64363a685f857556dc268` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.663370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06ae2f9cbf64363a685f857556dc268)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:16.668469 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1294/2580] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:16.668639 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:16.670574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4e597-12e7-48c4-8944-4797506d2175 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.686901 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4e597-12e7-48c4-8944-4797506d2175 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:16.695814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227e5ac-5004-46ab-98d2-d3db9d8290df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:16.697332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227e5ac-5004-46ab-98d2-d3db9d8290df admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.697689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227e5ac-5004-46ab-98d2-d3db9d8290df admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.698045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227e5ac-5004-46ab-98d2-d3db9d8290df admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.698824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227e5ac-5004-46ab-98d2-d3db9d8290df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:16.701693 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227e5ac-5004-46ab-98d2-d3db9d8290df admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:16.733308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:16.750907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1295/2581] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:16 2022] POST /identity/v3/projects => generated 399 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:16.759097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227e5ac-5004-46ab-98d2-d3db9d8290df admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.809841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06ae2f9cbf64363a685f857556dc268', 'request_id': 'req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:16.828409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06ae2f9cbf64363a685f857556dc268', 'request_id': 'req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:16.850810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06ae2f9cbf64363a685f857556dc268', 'request_id': 'req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:16.885606 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] The token cache is being invalidate because project a06ae2f9cbf64363a685f857556dc268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:16.886042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:16.886641 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:16.891380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1287/2582] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:16 2022] DELETE /identity/v3/projects/a06ae2f9cbf64363a685f857556dc268 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:16.897186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.897399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227e5ac-5004-46ab-98d2-d3db9d8290df admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:16.901417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:16.909277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:16.909880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.911143 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.911554 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.912395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:16.918818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:16.946944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1296/2583] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:16 2022] POST /identity/v3/users => generated 306 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:16.950689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:16.954189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:16.979323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb619c8b-bfbc-448e-92fa-fae1fa271348 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:16.986742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028d32-860e-4fa0-89ef-bb2db9c9d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:16.987563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028d32-860e-4fa0-89ef-bb2db9c9d18c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:16.987956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028d32-860e-4fa0-89ef-bb2db9c9d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:16.988350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028d32-860e-4fa0-89ef-bb2db9c9d18c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:16.989907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e028d32-860e-4fa0-89ef-bb2db9c9d18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:16.992131 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e028d32-860e-4fa0-89ef-bb2db9c9d18c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:17.005721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1297/2584] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:17.010806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028d32-860e-4fa0-89ef-bb2db9c9d18c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:17.025434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028d32-860e-4fa0-89ef-bb2db9c9d18c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:17.039048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38797bb1-99ad-4d7f-98c0-e52bacb74e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:17.039457 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38797bb1-99ad-4d7f-98c0-e52bacb74e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:17.039854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38797bb1-99ad-4d7f-98c0-e52bacb74e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:17.040223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38797bb1-99ad-4d7f-98c0-e52bacb74e52 admin admin] PATH_INFO: `/v3/projects/367f865d8dec4999aecd0fcd7eddbba3/users/b8d0212e62e24de8b113443719c4f06e/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:17.067388 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38797bb1-99ad-4d7f-98c0-e52bacb74e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=367f865d8dec4999aecd0fcd7eddbba3, user_id=b8d0212e62e24de8b113443719c4f06e, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:17.072784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38797bb1-99ad-4d7f-98c0-e52bacb74e52 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:17.097742 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1298/2585] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:17 2022] PUT /identity/v3/projects/367f865d8dec4999aecd0fcd7eddbba3/users/b8d0212e62e24de8b113443719c4f06e/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:17.100009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38797bb1-99ad-4d7f-98c0-e52bacb74e52 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:17.115386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82d02a-47ae-4d46-9a24-67a24d5ae9cc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:17.121115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:17.246519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38797bb1-99ad-4d7f-98c0-e52bacb74e52 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:17.260415 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d0702-cf87-4915-ba3f-a34516facf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:17.262570 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d0702-cf87-4915-ba3f-a34516facf7d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:17.263912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d0702-cf87-4915-ba3f-a34516facf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:17.265307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d0702-cf87-4915-ba3f-a34516facf7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:17.268594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d0702-cf87-4915-ba3f-a34516facf7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:17.282726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d0702-cf87-4915-ba3f-a34516facf7d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:17.307732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:17.308697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:17.308957 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:17.309238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] PATH_INFO: `/v3/users/cee2d2b314b94f24b5cbf07eb57eb831` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:17.311842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee2d2b314b94f24b5cbf07eb57eb831)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:17.317041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:17.320751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1299/2586] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:17.323028 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d0702-cf87-4915-ba3f-a34516facf7d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:17.347451 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d0702-cf87-4915-ba3f-a34516facf7d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:17.362641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e876857-0052-4d6d-9965-122d982a7441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:17.364018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e876857-0052-4d6d-9965-122d982a7441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:17.364603 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e876857-0052-4d6d-9965-122d982a7441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:17.365199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e876857-0052-4d6d-9965-122d982a7441 admin admin] PATH_INFO: `/v3/projects/367f865d8dec4999aecd0fcd7eddbba3/users/b8d0212e62e24de8b113443719c4f06e/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:17.368008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e876857-0052-4d6d-9965-122d982a7441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=367f865d8dec4999aecd0fcd7eddbba3, user_id=b8d0212e62e24de8b113443719c4f06e, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:17.371508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e876857-0052-4d6d-9965-122d982a7441 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:17.390726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1300/2587] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:17 2022] PUT /identity/v3/projects/367f865d8dec4999aecd0fcd7eddbba3/users/b8d0212e62e24de8b113443719c4f06e/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:17.405546 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a96ac-bffd-48a7-936c-3474f6f7341c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:17.406117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a96ac-bffd-48a7-936c-3474f6f7341c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:17.406692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a96ac-bffd-48a7-936c-3474f6f7341c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:17.448713 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-db4a96ac-bffd-48a7-936c-3474f6f7341c None None] MFA Rules not processed for user `b8d0212e62e24de8b113443719c4f06e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:17.450631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db4a96ac-bffd-48a7-936c-3474f6f7341c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:17.460232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee2d2b314b94f24b5cbf07eb57eb831'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:17.481522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:17.488811 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee2d2b314b94f24b5cbf07eb57eb831'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:17.489625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee2d2b314b94f24b5cbf07eb57eb831'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:17.509307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee2d2b314b94f24b5cbf07eb57eb831'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:17.513050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee2d2b314b94f24b5cbf07eb57eb831'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:17.556139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1288/2588] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:16 2022] DELETE /identity/v3/users/cee2d2b314b94f24b5cbf07eb57eb831 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:17.641312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1301/2589] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:17 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:17.653875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:17.655357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:17.802204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a2f5-1835-4a52-88e0-692a94db6708 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:17.811043 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:17.811713 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:17.812005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:17.812340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:17.917723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:17.919263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:17.932824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1289/2590] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:17 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:18.065677 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a96ac-bffd-48a7-936c-3474f6f7341c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:18.090680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:18.092742 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:18.174148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230144c-ecbc-4659-a4c9-ce7ce0587836 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:18.179025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:18.180047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.180468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.180845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] PATH_INFO: `/v3/projects/d673597c2e184f62b8963b529cc358df` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.182188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d673597c2e184f62b8963b529cc358df)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:18.184237 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:18.211507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a96ac-bffd-48a7-936c-3474f6f7341c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:18.222627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfdc2b-d87f-4e8b-9e20-5470730d33c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:18.224252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc2b-d87f-4e8b-9e20-5470730d33c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.224729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc2b-d87f-4e8b-9e20-5470730d33c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.225078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc2b-d87f-4e8b-9e20-5470730d33c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.230075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a1bfdc2b-d87f-4e8b-9e20-5470730d33c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:18.234994 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:18.274498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd673597c2e184f62b8963b529cc358df', 'request_id': 'req-5a1df07f-59e4-474a-acdd-48b40772d830'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:18.284213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd673597c2e184f62b8963b529cc358df', 'request_id': 'req-5a1df07f-59e4-474a-acdd-48b40772d830'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:18.293818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd673597c2e184f62b8963b529cc358df', 'request_id': 'req-5a1df07f-59e4-474a-acdd-48b40772d830'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:18.303841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:18.320345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] The token cache is being invalidate because project d673597c2e184f62b8963b529cc358df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:18.320800 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:18.321988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5a1df07f-59e4-474a-acdd-48b40772d830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:18.328316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1290/2591] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:18 2022] DELETE /identity/v3/projects/d673597c2e184f62b8963b529cc358df => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:18.330532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8018f1a-329e-484c-9930-d32a18a6a0fd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.330731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8018f1a-329e-484c-9930-d32a18a6a0fd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.331023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8018f1a-329e-484c-9930-d32a18a6a0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.345269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfdc2b-d87f-4e8b-9e20-5470730d33c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:18.347254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfdc2b-d87f-4e8b-9e20-5470730d33c0 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:18.353635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a8018f1a-329e-484c-9930-d32a18a6a0fd None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:18.355182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a8018f1a-329e-484c-9930-d32a18a6a0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:18.371092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1302/2592] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:18.471415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1291/2593] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:18.480492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518470bc-e76d-459f-9eb9-32bf96e6e519 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.480866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518470bc-e76d-459f-9eb9-32bf96e6e519 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.481238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518470bc-e76d-459f-9eb9-32bf96e6e519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.505406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-518470bc-e76d-459f-9eb9-32bf96e6e519 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:18.506347 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-518470bc-e76d-459f-9eb9-32bf96e6e519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:18.516605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1303/2594] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:18.522363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8018f1a-329e-484c-9930-d32a18a6a0fd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:18.538568 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8018f1a-329e-484c-9930-d32a18a6a0fd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:18.544199 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:18.545359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.545359 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.545578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.546313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:18.546627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:18.549257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:18.587171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1292/2595] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:18.587384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518470bc-e76d-459f-9eb9-32bf96e6e519 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:18.600640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518470bc-e76d-459f-9eb9-32bf96e6e519 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:18.615752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614784c1-08e3-4a51-a25e-206fff565748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:18.616484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614784c1-08e3-4a51-a25e-206fff565748 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.616767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614784c1-08e3-4a51-a25e-206fff565748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.617046 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614784c1-08e3-4a51-a25e-206fff565748 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.617761 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614784c1-08e3-4a51-a25e-206fff565748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:18.619743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614784c1-08e3-4a51-a25e-206fff565748 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:18.686760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1304/2596] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:18 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:18.688353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:18.707195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1a0f4-7d46-4c45-ad4a-bb378e6ba68a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:18.849916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16331e17-564d-483c-9a35-e876f582d01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:18.850690 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16331e17-564d-483c-9a35-e876f582d01e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.851015 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16331e17-564d-483c-9a35-e876f582d01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.851219 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16331e17-564d-483c-9a35-e876f582d01e admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.853340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16331e17-564d-483c-9a35-e876f582d01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:18.855712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16331e17-564d-483c-9a35-e876f582d01e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:18.893307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1293/2597] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:18 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:18.897579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614784c1-08e3-4a51-a25e-206fff565748 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:18.911823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614784c1-08e3-4a51-a25e-206fff565748 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:18.917604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03670eb3-65f6-4c21-8d3a-a32154a41d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:18.918422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03670eb3-65f6-4c21-8d3a-a32154a41d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.918692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03670eb3-65f6-4c21-8d3a-a32154a41d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.919034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03670eb3-65f6-4c21-8d3a-a32154a41d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.919844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03670eb3-65f6-4c21-8d3a-a32154a41d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:18.921486 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03670eb3-65f6-4c21-8d3a-a32154a41d1c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:18.938583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1305/2598] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:18.940460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16331e17-564d-483c-9a35-e876f582d01e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:18.952083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16331e17-564d-483c-9a35-e876f582d01e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:18.958891 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af44ca8-ed40-4178-bf4e-3ad0b51ca03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:18.959838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af44ca8-ed40-4178-bf4e-3ad0b51ca03f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.960265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af44ca8-ed40-4178-bf4e-3ad0b51ca03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.960647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af44ca8-ed40-4178-bf4e-3ad0b51ca03f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:18.961599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af44ca8-ed40-4178-bf4e-3ad0b51ca03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:18.963322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af44ca8-ed40-4178-bf4e-3ad0b51ca03f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:18.974858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1294/2599] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:18.979315 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03670eb3-65f6-4c21-8d3a-a32154a41d1c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:18.990222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03670eb3-65f6-4c21-8d3a-a32154a41d1c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:18.995518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:18.996766 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:18.997241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:18.997723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] PATH_INFO: `/v3/projects/0b9ea0feaee34c639f228a4f320f1aa3/users/f869410b53b14949a0f066e0e4163a45/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:19.016850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9ea0feaee34c639f228a4f320f1aa3, user_id=f869410b53b14949a0f066e0e4163a45, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:19.021619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:19.041474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1306/2600] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:18 2022] PUT /identity/v3/projects/0b9ea0feaee34c639f228a4f320f1aa3/users/f869410b53b14949a0f066e0e4163a45/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:19.048599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb4d1c-7387-493c-b92e-74bf5143bbfc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:19.049042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb4d1c-7387-493c-b92e-74bf5143bbfc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:19.049429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb4d1c-7387-493c-b92e-74bf5143bbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:19.095584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-aabb4d1c-7387-493c-b92e-74bf5143bbfc None None] MFA Rules not processed for user `f869410b53b14949a0f066e0e4163a45`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:19.097281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-aabb4d1c-7387-493c-b92e-74bf5143bbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:19.110474 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:19.173093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1295/2601] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:19.185690 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:19.187675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:19.252541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb01f47-5b94-464a-9c90-345bc4c90340 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:19.256748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:19.257398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:19.257683 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:19.257960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:19.267230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:19.269539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:19.280527 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1307/2602] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:20.664675 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6511a87-283a-4c9f-b44a-baf68c339d98 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:20.665226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6511a87-283a-4c9f-b44a-baf68c339d98 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:20.665226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6511a87-283a-4c9f-b44a-baf68c339d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:20.685125 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b6511a87-283a-4c9f-b44a-baf68c339d98 None None] MFA Rules not processed for user `f869410b53b14949a0f066e0e4163a45`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:20.686350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b6511a87-283a-4c9f-b44a-baf68c339d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:20.698452 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1296/2603] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:11:22.903942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:22.905528 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:23.008405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1a13-ea4c-4528-8afd-b9dab63ef005 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:23.015566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:23.015566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:23.015566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:23.015566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] PATH_INFO: `/v3/users/05e17fe44239485582511be5fc02e31c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:23.016201 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e17fe44239485582511be5fc02e31c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:23.019462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:23.039031 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:23.089193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e17fe44239485582511be5fc02e31c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:23.109615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e17fe44239485582511be5fc02e31c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:23.110656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e17fe44239485582511be5fc02e31c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:23.111496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e17fe44239485582511be5fc02e31c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:23.122331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e17fe44239485582511be5fc02e31c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:23.137905 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1308/2604] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:22 2022] DELETE /identity/v3/users/05e17fe44239485582511be5fc02e31c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:23.143173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a75c7d-4825-479c-b568-01af97f27a51 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:23.143997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a75c7d-4825-479c-b568-01af97f27a51 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:23.143997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a75c7d-4825-479c-b568-01af97f27a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:23.163657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-31a75c7d-4825-479c-b568-01af97f27a51 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:23.164931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-31a75c7d-4825-479c-b568-01af97f27a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:23.234841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1297/2605] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:11:23.245210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:23.247467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:23.312096 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2e3f0-66ec-40ff-a7bb-48150e1413f4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:23.317039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232b3df-599f-4831-af2c-d4d0603e81d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:23.317856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232b3df-599f-4831-af2c-d4d0603e81d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:23.318153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232b3df-599f-4831-af2c-d4d0603e81d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:23.318430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232b3df-599f-4831-af2c-d4d0603e81d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:23.328297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232b3df-599f-4831-af2c-d4d0603e81d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:23.330382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232b3df-599f-4831-af2c-d4d0603e81d5 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:23.342175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1309/2606] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:23.779586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a75c7d-4825-479c-b568-01af97f27a51 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:23.781174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-31a75c7d-4825-479c-b568-01af97f27a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:23.795301 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a75c7d-4825-479c-b568-01af97f27a51 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:23.801158 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:23.802000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:23.802294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:23.802656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] PATH_INFO: `/v3/projects/91cf9917f195476c9b9561c94c31255d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:23.803988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91cf9917f195476c9b9561c94c31255d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:23.806313 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:23.868097 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91cf9917f195476c9b9561c94c31255d', 'request_id': 'req-09809538-fdb3-4652-a041-20a2f56aa328'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:23.878647 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91cf9917f195476c9b9561c94c31255d', 'request_id': 'req-09809538-fdb3-4652-a041-20a2f56aa328'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:23.888783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91cf9917f195476c9b9561c94c31255d', 'request_id': 'req-09809538-fdb3-4652-a041-20a2f56aa328'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.037991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] The token cache is being invalidate because project 91cf9917f195476c9b9561c94c31255d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:24.038312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.039051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.041151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1298/2607] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:23 2022] DELETE /identity/v3/projects/91cf9917f195476c9b9561c94c31255d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:24.046789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232b3df-599f-4831-af2c-d4d0603e81d5 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:24.057524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232b3df-599f-4831-af2c-d4d0603e81d5 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:24.115233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:24.116102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:24.116418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:24.116695 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] PATH_INFO: `/v3/users/b8d0212e62e24de8b113443719c4f06e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:24.117894 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d0212e62e24de8b113443719c4f06e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:24.119863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:24.160539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:24.170889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d0212e62e24de8b113443719c4f06e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.185255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d0212e62e24de8b113443719c4f06e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.185980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d0212e62e24de8b113443719c4f06e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.186597 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d0212e62e24de8b113443719c4f06e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.194262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1ddb1075-3f70-46f5-9e6f-1c4556e0aa95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d0212e62e24de8b113443719c4f06e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.208722 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1310/2608] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:24 2022] DELETE /identity/v3/users/b8d0212e62e24de8b113443719c4f06e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:24.527081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:24.528973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:24.550681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809538-fdb3-4652-a041-20a2f56aa328 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:24.589137 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:24.625763 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:24.626851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:24.627351 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:24.627636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] PATH_INFO: `/v3/projects/367f865d8dec4999aecd0fcd7eddbba3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:24.629014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367f865d8dec4999aecd0fcd7eddbba3)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:24.631120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:24.691990 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367f865d8dec4999aecd0fcd7eddbba3', 'request_id': 'req-48f9677a-65a7-48e0-bdb9-259226ff4997'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.699266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367f865d8dec4999aecd0fcd7eddbba3', 'request_id': 'req-48f9677a-65a7-48e0-bdb9-259226ff4997'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.706973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367f865d8dec4999aecd0fcd7eddbba3', 'request_id': 'req-48f9677a-65a7-48e0-bdb9-259226ff4997'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.725707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] The token cache is being invalidate because project 367f865d8dec4999aecd0fcd7eddbba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:24.726078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.727092 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48f9677a-65a7-48e0-bdb9-259226ff4997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:24.729349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1299/2609] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:24 2022] DELETE /identity/v3/projects/367f865d8dec4999aecd0fcd7eddbba3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:24.735152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671bf9-0ee0-493c-8198-985f0ef16e49 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:24.735345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671bf9-0ee0-493c-8198-985f0ef16e49 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:24.735835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671bf9-0ee0-493c-8198-985f0ef16e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:24.756265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7e671bf9-0ee0-493c-8198-985f0ef16e49 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:24.757504 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7e671bf9-0ee0-493c-8198-985f0ef16e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:24.829623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1311/2610] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:24.841762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6fe26-632f-4673-86bb-c7fa67f9a714 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:24.842050 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6fe26-632f-4673-86bb-c7fa67f9a714 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:24.842341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6fe26-632f-4673-86bb-c7fa67f9a714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:24.860920 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67b6fe26-632f-4673-86bb-c7fa67f9a714 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:24.862132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67b6fe26-632f-4673-86bb-c7fa67f9a714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:24.872104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1300/2611] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:11:24.877395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e671bf9-0ee0-493c-8198-985f0ef16e49 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:24.890423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e671bf9-0ee0-493c-8198-985f0ef16e49 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:24.898772 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:24.900089 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:24.900610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:24.900896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:24.901653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:24.901974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:24.903837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:24.918980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1312/2612] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:24.924992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6fe26-632f-4673-86bb-c7fa67f9a714 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:24.935597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6fe26-632f-4673-86bb-c7fa67f9a714 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:24.941415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6e6a1-db42-428c-a5e0-686c1b1f9cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:24.942159 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6e6a1-db42-428c-a5e0-686c1b1f9cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:24.942371 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6e6a1-db42-428c-a5e0-686c1b1f9cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:24.942652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6e6a1-db42-428c-a5e0-686c1b1f9cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:24.943370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6e6a1-db42-428c-a5e0-686c1b1f9cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:24.945354 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6e6a1-db42-428c-a5e0-686c1b1f9cf5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:24.979387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1301/2613] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:24 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:24.984109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:24.997520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170dd79-a26a-4c4e-a98e-0489a3041ac3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.065563 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04802d51-351d-4da2-9bb5-393bbedd24fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.066277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04802d51-351d-4da2-9bb5-393bbedd24fd admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.066513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04802d51-351d-4da2-9bb5-393bbedd24fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.066795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04802d51-351d-4da2-9bb5-393bbedd24fd admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.067497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04802d51-351d-4da2-9bb5-393bbedd24fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.069457 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04802d51-351d-4da2-9bb5-393bbedd24fd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.093005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1313/2614] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:24 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:25.097231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6e6a1-db42-428c-a5e0-686c1b1f9cf5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.109959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6e6a1-db42-428c-a5e0-686c1b1f9cf5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.115574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7672a-227f-4ea6-bb9e-bdbb521f70a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.116278 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7672a-227f-4ea6-bb9e-bdbb521f70a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.116563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7672a-227f-4ea6-bb9e-bdbb521f70a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.116842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7672a-227f-4ea6-bb9e-bdbb521f70a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.117570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7672a-227f-4ea6-bb9e-bdbb521f70a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.119168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7672a-227f-4ea6-bb9e-bdbb521f70a5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.132299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1302/2615] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:25.136381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04802d51-351d-4da2-9bb5-393bbedd24fd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.147721 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04802d51-351d-4da2-9bb5-393bbedd24fd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.153311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c3eea-be64-48b9-86db-e54f484855a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.153992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c3eea-be64-48b9-86db-e54f484855a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.154297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c3eea-be64-48b9-86db-e54f484855a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.154569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c3eea-be64-48b9-86db-e54f484855a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.155914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c3eea-be64-48b9-86db-e54f484855a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.158085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c3eea-be64-48b9-86db-e54f484855a4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.171876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1314/2616] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:25.177171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7672a-227f-4ea6-bb9e-bdbb521f70a5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.187931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7672a-227f-4ea6-bb9e-bdbb521f70a5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.194660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64964430-e9a2-4481-8c06-3087aa608c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.195612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964430-e9a2-4481-8c06-3087aa608c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.195902 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964430-e9a2-4481-8c06-3087aa608c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.196279 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64964430-e9a2-4481-8c06-3087aa608c74 admin admin] PATH_INFO: `/v3/projects/744b4dc4307e4762a1dc2e02a7449f95/users/12065614cb0a4ccd934ebe428c6520cb/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.219947 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64964430-e9a2-4481-8c06-3087aa608c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744b4dc4307e4762a1dc2e02a7449f95, user_id=12065614cb0a4ccd934ebe428c6520cb, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.224473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64964430-e9a2-4481-8c06-3087aa608c74 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.240986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1303/2617] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:25 2022] PUT /identity/v3/projects/744b4dc4307e4762a1dc2e02a7449f95/users/12065614cb0a4ccd934ebe428c6520cb/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:25.251161 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79a72-8716-4cfb-8513-0bf614179bb5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.252017 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79a72-8716-4cfb-8513-0bf614179bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.252349 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79a72-8716-4cfb-8513-0bf614179bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.279090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-66a79a72-8716-4cfb-8513-0bf614179bb5 None None] MFA Rules not processed for user `12065614cb0a4ccd934ebe428c6520cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:25.280359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-66a79a72-8716-4cfb-8513-0bf614179bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:25.289754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:25.348575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1315/2618] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:25.353061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64964430-e9a2-4481-8c06-3087aa608c74 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.421540 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64964430-e9a2-4481-8c06-3087aa608c74 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.428415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0f13e-5d2b-4351-99eb-4c3d55656b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.429100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0f13e-5d2b-4351-99eb-4c3d55656b16 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.429385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0f13e-5d2b-4351-99eb-4c3d55656b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.429663 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0f13e-5d2b-4351-99eb-4c3d55656b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.430397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0f13e-5d2b-4351-99eb-4c3d55656b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.433492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0f13e-5d2b-4351-99eb-4c3d55656b16 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.440023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:25.467295 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1304/2619] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:25 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:11:25.475267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79a72-8716-4cfb-8513-0bf614179bb5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.488115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79a72-8716-4cfb-8513-0bf614179bb5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.550648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85ced-3988-4c1b-8811-a800a9f64be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.551333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85ced-3988-4c1b-8811-a800a9f64be3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.551669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85ced-3988-4c1b-8811-a800a9f64be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.551902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b85ced-3988-4c1b-8811-a800a9f64be3 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.552651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b85ced-3988-4c1b-8811-a800a9f64be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.554590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b85ced-3988-4c1b-8811-a800a9f64be3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.575321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1316/2620] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:25 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:25.579091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0f13e-5d2b-4351-99eb-4c3d55656b16 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.590304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0f13e-5d2b-4351-99eb-4c3d55656b16 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.595225 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92407a13-0ee3-4077-8f74-7f24d86f0691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.595983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92407a13-0ee3-4077-8f74-7f24d86f0691 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.596308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92407a13-0ee3-4077-8f74-7f24d86f0691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.596588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92407a13-0ee3-4077-8f74-7f24d86f0691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.597316 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92407a13-0ee3-4077-8f74-7f24d86f0691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.598918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92407a13-0ee3-4077-8f74-7f24d86f0691 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.610082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1305/2621] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:25.614521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85ced-3988-4c1b-8811-a800a9f64be3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.624496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b85ced-3988-4c1b-8811-a800a9f64be3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.629715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6532f-2861-46a6-b506-670c6f0b70aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.630748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6532f-2861-46a6-b506-670c6f0b70aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.631039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6532f-2861-46a6-b506-670c6f0b70aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.631322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6532f-2861-46a6-b506-670c6f0b70aa admin admin] PATH_INFO: `/v3/projects/5c0a935467c046cc8c07894310eefac8/users/4c0521da86b0446ba14e1c846a231b7b/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.648725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6532f-2861-46a6-b506-670c6f0b70aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0a935467c046cc8c07894310eefac8, user_id=4c0521da86b0446ba14e1c846a231b7b, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.651434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6532f-2861-46a6-b506-670c6f0b70aa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.665153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1317/2622] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:25 2022] PUT /identity/v3/projects/5c0a935467c046cc8c07894310eefac8/users/4c0521da86b0446ba14e1c846a231b7b/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:25.670077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92407a13-0ee3-4077-8f74-7f24d86f0691 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.688521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92407a13-0ee3-4077-8f74-7f24d86f0691 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.780106 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106575-9071-4f4d-b9a3-45de6c1d9691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.780828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106575-9071-4f4d-b9a3-45de6c1d9691 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.781139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106575-9071-4f4d-b9a3-45de6c1d9691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.781424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106575-9071-4f4d-b9a3-45de6c1d9691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.782170 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05106575-9071-4f4d-b9a3-45de6c1d9691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.783965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05106575-9071-4f4d-b9a3-45de6c1d9691 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.795142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1306/2623] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:25.799837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6532f-2861-46a6-b506-670c6f0b70aa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.811933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6532f-2861-46a6-b506-670c6f0b70aa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:25.817555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:25.818442 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.818727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.819000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] PATH_INFO: `/v3/projects/5c0a935467c046cc8c07894310eefac8/users/4c0521da86b0446ba14e1c846a231b7b/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.821064 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0a935467c046cc8c07894310eefac8, user_id=4c0521da86b0446ba14e1c846a231b7b, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:25.823678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:25.836298 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1318/2624] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:25 2022] PUT /identity/v3/projects/5c0a935467c046cc8c07894310eefac8/users/4c0521da86b0446ba14e1c846a231b7b/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:25.846208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb77fbd-2f0c-499c-80b0-be0112da426a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:25.846532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb77fbd-2f0c-499c-80b0-be0112da426a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:25.846819 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb77fbd-2f0c-499c-80b0-be0112da426a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:25.879293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ecb77fbd-2f0c-499c-80b0-be0112da426a None None] MFA Rules not processed for user `4c0521da86b0446ba14e1c846a231b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:25.880633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ecb77fbd-2f0c-499c-80b0-be0112da426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:25.968480 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1307/2625] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:25.979956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:25.981671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:26.046456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdd7c4-2e7b-4378-90be-6503800d1912 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:26.051122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95078546-09f7-43fe-b9a8-786913f23f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:26.051781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95078546-09f7-43fe-b9a8-786913f23f1b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:26.052079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95078546-09f7-43fe-b9a8-786913f23f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:26.052426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95078546-09f7-43fe-b9a8-786913f23f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:26.062217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95078546-09f7-43fe-b9a8-786913f23f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:26.063752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95078546-09f7-43fe-b9a8-786913f23f1b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:26.073760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1319/2626] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:11:26.213176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb77fbd-2f0c-499c-80b0-be0112da426a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:26.270960 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:26.277836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb77fbd-2f0c-499c-80b0-be0112da426a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:26.282914 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:26.283796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:26.284072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:26.284382 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] PATH_INFO: `/v3/users/12065614cb0a4ccd934ebe428c6520cb` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:26.285579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12065614cb0a4ccd934ebe428c6520cb)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:26.287510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:26.335669 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12065614cb0a4ccd934ebe428c6520cb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:26.347852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12065614cb0a4ccd934ebe428c6520cb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:26.348559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12065614cb0a4ccd934ebe428c6520cb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:26.355322 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12065614cb0a4ccd934ebe428c6520cb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:26.355963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12065614cb0a4ccd934ebe428c6520cb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:26.370039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1308/2627] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:26 2022] DELETE /identity/v3/users/12065614cb0a4ccd934ebe428c6520cb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:26.374566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e79a-3848-467d-8b57-ba3749a1d231 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:26.374860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e79a-3848-467d-8b57-ba3749a1d231 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:26.375146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e79a-3848-467d-8b57-ba3749a1d231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:26.383870 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:26.397215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ee50e79a-3848-467d-8b57-ba3749a1d231 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:26.398412 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ee50e79a-3848-467d-8b57-ba3749a1d231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:26.463508 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1320/2628] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:26.473441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:26.474845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:26.536066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6aae6-3374-413b-9310-45dec7813e21 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:26.539814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bea63-8a5a-4893-ae57-d59308e0debd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:26.540781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bea63-8a5a-4893-ae57-d59308e0debd service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:26.541072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bea63-8a5a-4893-ae57-d59308e0debd service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:26.541356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bea63-8a5a-4893-ae57-d59308e0debd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:26.551822 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bea63-8a5a-4893-ae57-d59308e0debd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:26.553682 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bea63-8a5a-4893-ae57-d59308e0debd service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:26.566001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1309/2629] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:27.147427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50e79a-3848-467d-8b57-ba3749a1d231 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:27.149037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ee50e79a-3848-467d-8b57-ba3749a1d231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:27.162890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50e79a-3848-467d-8b57-ba3749a1d231 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:27.167577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:27.168480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:27.168764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:27.169648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] PATH_INFO: `/v3/projects/744b4dc4307e4762a1dc2e02a7449f95` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:27.170960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=744b4dc4307e4762a1dc2e02a7449f95)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:27.173769 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:27.260011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744b4dc4307e4762a1dc2e02a7449f95', 'request_id': 'req-ea127aec-bec9-4f75-b4ce-d30698e344d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.269638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744b4dc4307e4762a1dc2e02a7449f95', 'request_id': 'req-ea127aec-bec9-4f75-b4ce-d30698e344d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.277751 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744b4dc4307e4762a1dc2e02a7449f95', 'request_id': 'req-ea127aec-bec9-4f75-b4ce-d30698e344d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.290627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] The token cache is being invalidate because project 744b4dc4307e4762a1dc2e02a7449f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:27.291920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.292891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.295119 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1321/2630] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:27 2022] DELETE /identity/v3/projects/744b4dc4307e4762a1dc2e02a7449f95 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:27.300049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bea63-8a5a-4893-ae57-d59308e0debd service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:27.312827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bea63-8a5a-4893-ae57-d59308e0debd service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:27.382005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:27.383117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:27.383427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:27.383903 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] PATH_INFO: `/v3/users/4c0521da86b0446ba14e1c846a231b7b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:27.385499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0521da86b0446ba14e1c846a231b7b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:27.387680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:27.415311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:27.458403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0521da86b0446ba14e1c846a231b7b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.472941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0521da86b0446ba14e1c846a231b7b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.473705 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0521da86b0446ba14e1c846a231b7b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.481239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0521da86b0446ba14e1c846a231b7b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.482000 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-748be136-f206-4434-8482-c76b1f755200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0521da86b0446ba14e1c846a231b7b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.495089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1310/2631] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:27 2022] DELETE /identity/v3/users/4c0521da86b0446ba14e1c846a231b7b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:27.789083 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:27.790541 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:27.818216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea127aec-bec9-4f75-b4ce-d30698e344d6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:27.891381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:27.893226 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:27.893589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:27.893940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] PATH_INFO: `/v3/projects/5c0a935467c046cc8c07894310eefac8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:27.895265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0a935467c046cc8c07894310eefac8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:27.897473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:27.943948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:27.963569 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0a935467c046cc8c07894310eefac8', 'request_id': 'req-d0a73ef4-5a0e-4875-8341-d1a762465c2d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.982616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0a935467c046cc8c07894310eefac8', 'request_id': 'req-d0a73ef4-5a0e-4875-8341-d1a762465c2d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:27.992331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0a935467c046cc8c07894310eefac8', 'request_id': 'req-d0a73ef4-5a0e-4875-8341-d1a762465c2d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:28.004488 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] The token cache is being invalidate because project 5c0a935467c046cc8c07894310eefac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:28.004784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:28.005461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0a73ef4-5a0e-4875-8341-d1a762465c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:28.007775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1322/2632] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:27 2022] DELETE /identity/v3/projects/5c0a935467c046cc8c07894310eefac8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:28.013259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465709-1f1c-4be4-875b-f73621d90159 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.013716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465709-1f1c-4be4-875b-f73621d90159 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.014190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465709-1f1c-4be4-875b-f73621d90159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.037116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9d465709-1f1c-4be4-875b-f73621d90159 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:28.038590 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9d465709-1f1c-4be4-875b-f73621d90159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:28.109703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1311/2633] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:28.118397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4ba3d-7ef5-47ba-82b8-9b4d6e99e402 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.118687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4ba3d-7ef5-47ba-82b8-9b4d6e99e402 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.119236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4ba3d-7ef5-47ba-82b8-9b4d6e99e402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.137631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d1c4ba3d-7ef5-47ba-82b8-9b4d6e99e402 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:28.138838 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d1c4ba3d-7ef5-47ba-82b8-9b4d6e99e402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:28.148352 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1323/2634] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:28.152646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465709-1f1c-4be4-875b-f73621d90159 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.163886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465709-1f1c-4be4-875b-f73621d90159 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:28.169155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:28.169831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.170115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.170387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.174039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:28.174039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:28.174039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:28.188228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1312/2635] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:28.199739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4ba3d-7ef5-47ba-82b8-9b4d6e99e402 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.214338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4ba3d-7ef5-47ba-82b8-9b4d6e99e402 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:28.219631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff0926-9f04-4665-8dd9-f306758f0d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:28.221075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0926-9f04-4665-8dd9-f306758f0d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.221369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0926-9f04-4665-8dd9-f306758f0d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.221647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0926-9f04-4665-8dd9-f306758f0d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.222366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff0926-9f04-4665-8dd9-f306758f0d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:28.224422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff0926-9f04-4665-8dd9-f306758f0d2c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:28.251538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1324/2636] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:28 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:28.256055 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.267297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd26565-9463-430f-89a0-ab0dd4995d75 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:28.328980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47eaa-b41b-4bc1-9715-058e01f50b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:28.330215 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47eaa-b41b-4bc1-9715-058e01f50b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.330455 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47eaa-b41b-4bc1-9715-058e01f50b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.330738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47eaa-b41b-4bc1-9715-058e01f50b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.331438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b47eaa-b41b-4bc1-9715-058e01f50b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:28.333423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b47eaa-b41b-4bc1-9715-058e01f50b20 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:28.355090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1313/2637] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:28 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:28.359159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff0926-9f04-4665-8dd9-f306758f0d2c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.371613 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff0926-9f04-4665-8dd9-f306758f0d2c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:28.376864 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fea548-fb91-4afa-8728-484f1d849cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:28.377502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fea548-fb91-4afa-8728-484f1d849cae admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.378629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fea548-fb91-4afa-8728-484f1d849cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.378984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fea548-fb91-4afa-8728-484f1d849cae admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.380012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fea548-fb91-4afa-8728-484f1d849cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:28.382286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fea548-fb91-4afa-8728-484f1d849cae admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:28.396571 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1325/2638] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:28.401039 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47eaa-b41b-4bc1-9715-058e01f50b20 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.412128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47eaa-b41b-4bc1-9715-058e01f50b20 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:28.417216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb5486-19b6-4d62-a89c-9ce14b4c3239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:28.417839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb5486-19b6-4d62-a89c-9ce14b4c3239 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.418128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb5486-19b6-4d62-a89c-9ce14b4c3239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.418408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb5486-19b6-4d62-a89c-9ce14b4c3239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.419150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb5486-19b6-4d62-a89c-9ce14b4c3239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:28.421133 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb5486-19b6-4d62-a89c-9ce14b4c3239 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:28.436238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1314/2639] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:28.440628 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fea548-fb91-4afa-8728-484f1d849cae admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.466946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fea548-fb91-4afa-8728-484f1d849cae admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:28.475610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a80df2-f110-4e46-9079-5f242cb468e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:28.476589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a80df2-f110-4e46-9079-5f242cb468e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.476883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a80df2-f110-4e46-9079-5f242cb468e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.477166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a80df2-f110-4e46-9079-5f242cb468e6 admin admin] PATH_INFO: `/v3/projects/6cfdf2f2d56841dc84e57a169c6e7d0e/users/fc1b02d5172c4f58a119f3df87c88e78/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.493235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a80df2-f110-4e46-9079-5f242cb468e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfdf2f2d56841dc84e57a169c6e7d0e, user_id=fc1b02d5172c4f58a119f3df87c88e78, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:28.495922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a80df2-f110-4e46-9079-5f242cb468e6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:28.512516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1326/2640] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:28 2022] PUT /identity/v3/projects/6cfdf2f2d56841dc84e57a169c6e7d0e/users/fc1b02d5172c4f58a119f3df87c88e78/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:28.524721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5796fe-8e25-468b-a52d-d1c0e5db30f8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.525041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5796fe-8e25-468b-a52d-d1c0e5db30f8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.525421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5796fe-8e25-468b-a52d-d1c0e5db30f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.531660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:28.561460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4f5796fe-8e25-468b-a52d-d1c0e5db30f8 None None] MFA Rules not processed for user `fc1b02d5172c4f58a119f3df87c88e78`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:28.562893 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4f5796fe-8e25-468b-a52d-d1c0e5db30f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:28.667660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1315/2641] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:28.673760 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a80df2-f110-4e46-9079-5f242cb468e6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.754129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a80df2-f110-4e46-9079-5f242cb468e6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:28.759211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64b1d0-b01a-4901-96db-f22f9c8f3942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:28.760006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64b1d0-b01a-4901-96db-f22f9c8f3942 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.760422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64b1d0-b01a-4901-96db-f22f9c8f3942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.760809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64b1d0-b01a-4901-96db-f22f9c8f3942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.761607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64b1d0-b01a-4901-96db-f22f9c8f3942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:28.763639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64b1d0-b01a-4901-96db-f22f9c8f3942 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:28.799876 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1327/2642] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:28 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:11:28.803997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5796fe-8e25-468b-a52d-d1c0e5db30f8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.822281 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5796fe-8e25-468b-a52d-d1c0e5db30f8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:28.938260 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616846e7-c110-450d-8cf1-860ffcfbfff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:28.939597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616846e7-c110-450d-8cf1-860ffcfbfff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:28.943755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616846e7-c110-450d-8cf1-860ffcfbfff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:28.944056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616846e7-c110-450d-8cf1-860ffcfbfff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:28.945175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616846e7-c110-450d-8cf1-860ffcfbfff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:28.948464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616846e7-c110-450d-8cf1-860ffcfbfff4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:28.977802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1316/2643] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:28 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:11:28.982267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64b1d0-b01a-4901-96db-f22f9c8f3942 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:28.998498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64b1d0-b01a-4901-96db-f22f9c8f3942 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:29.010932 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878ac0c-769f-49e1-ad0a-6c897d175cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:29.012116 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878ac0c-769f-49e1-ad0a-6c897d175cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.012471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878ac0c-769f-49e1-ad0a-6c897d175cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.012715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878ac0c-769f-49e1-ad0a-6c897d175cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.013522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878ac0c-769f-49e1-ad0a-6c897d175cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:29.027219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878ac0c-769f-49e1-ad0a-6c897d175cfc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:29.057758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1328/2644] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:29.061448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616846e7-c110-450d-8cf1-860ffcfbfff4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:29.083187 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616846e7-c110-450d-8cf1-860ffcfbfff4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:29.095701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a5011-ff8a-45cb-914b-6699c96227c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:29.096597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5011-ff8a-45cb-914b-6699c96227c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.097022 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5011-ff8a-45cb-914b-6699c96227c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.097203 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5011-ff8a-45cb-914b-6699c96227c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.098045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a5011-ff8a-45cb-914b-6699c96227c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:29.099633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a5011-ff8a-45cb-914b-6699c96227c8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:29.136609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1317/2645] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:29.138494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878ac0c-769f-49e1-ad0a-6c897d175cfc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:29.158846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878ac0c-769f-49e1-ad0a-6c897d175cfc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:29.178949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602816f0-8f57-48e4-9539-f6d1f60ab474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:29.182133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602816f0-8f57-48e4-9539-f6d1f60ab474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.185057 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602816f0-8f57-48e4-9539-f6d1f60ab474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.185596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602816f0-8f57-48e4-9539-f6d1f60ab474 admin admin] PATH_INFO: `/v3/projects/6e2e6231cf4c4be9808995203d7bc5a5/users/8b3b5792efc14a0e9f1f832c5e152ec3/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.246795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602816f0-8f57-48e4-9539-f6d1f60ab474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2e6231cf4c4be9808995203d7bc5a5, user_id=8b3b5792efc14a0e9f1f832c5e152ec3, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:29.249574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602816f0-8f57-48e4-9539-f6d1f60ab474 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:29.295855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1329/2646] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:29 2022] PUT /identity/v3/projects/6e2e6231cf4c4be9808995203d7bc5a5/users/8b3b5792efc14a0e9f1f832c5e152ec3/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:29.303126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993d2f-4026-4885-801a-569561e190e1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.303457 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993d2f-4026-4885-801a-569561e190e1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.303693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2993d2f-4026-4885-801a-569561e190e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.341434 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a2993d2f-4026-4885-801a-569561e190e1 None None] MFA Rules not processed for user `8b3b5792efc14a0e9f1f832c5e152ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:29.344426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a2993d2f-4026-4885-801a-569561e190e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:29.436040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:29.442421 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1318/2647] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:29.446951 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602816f0-8f57-48e4-9539-f6d1f60ab474 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:29.453418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:29.526367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602816f0-8f57-48e4-9539-f6d1f60ab474 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:29.535072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a606bb-1d11-462d-ad07-7b4d18b9cfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:29.535892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a606bb-1d11-462d-ad07-7b4d18b9cfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.536280 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a606bb-1d11-462d-ad07-7b4d18b9cfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.537145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a606bb-1d11-462d-ad07-7b4d18b9cfa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.537467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a606bb-1d11-462d-ad07-7b4d18b9cfa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:29.539523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a606bb-1d11-462d-ad07-7b4d18b9cfa0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:29.589893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1330/2648] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:29 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:29.594299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2993d2f-4026-4885-801a-569561e190e1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:29.620829 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2993d2f-4026-4885-801a-569561e190e1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:29.699496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73201edb-0a8e-4c2d-bb1b-8825df90b665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:29.700353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73201edb-0a8e-4c2d-bb1b-8825df90b665 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.701249 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73201edb-0a8e-4c2d-bb1b-8825df90b665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.701636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73201edb-0a8e-4c2d-bb1b-8825df90b665 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.702662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73201edb-0a8e-4c2d-bb1b-8825df90b665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:29.705589 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73201edb-0a8e-4c2d-bb1b-8825df90b665 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:29.760971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1319/2649] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:29 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:29.766175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a606bb-1d11-462d-ad07-7b4d18b9cfa0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:29.783271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a606bb-1d11-462d-ad07-7b4d18b9cfa0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:29.805271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793b7b-55dd-4a59-863c-8d5df8fdefb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:29.806367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793b7b-55dd-4a59-863c-8d5df8fdefb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.806866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793b7b-55dd-4a59-863c-8d5df8fdefb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.808795 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793b7b-55dd-4a59-863c-8d5df8fdefb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.809875 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f793b7b-55dd-4a59-863c-8d5df8fdefb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:29.812107 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f793b7b-55dd-4a59-863c-8d5df8fdefb2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:29.826307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1331/2650] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:29.831884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73201edb-0a8e-4c2d-bb1b-8825df90b665 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:29.844077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73201edb-0a8e-4c2d-bb1b-8825df90b665 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:29.850489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34182c89-6080-43dd-9e26-f86b5012594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:29.851598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34182c89-6080-43dd-9e26-f86b5012594c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.852036 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34182c89-6080-43dd-9e26-f86b5012594c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.852355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34182c89-6080-43dd-9e26-f86b5012594c admin admin] PATH_INFO: `/v3/projects/fc6af8a2f6a749c199fcfb659b3cdd33/users/3b47e1d2ec2d47d5a8819869b432f228/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.873802 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34182c89-6080-43dd-9e26-f86b5012594c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6af8a2f6a749c199fcfb659b3cdd33, user_id=3b47e1d2ec2d47d5a8819869b432f228, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:29.877877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34182c89-6080-43dd-9e26-f86b5012594c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:29.900558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1320/2651] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:29 2022] PUT /identity/v3/projects/fc6af8a2f6a749c199fcfb659b3cdd33/users/3b47e1d2ec2d47d5a8819869b432f228/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:29.902547 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793b7b-55dd-4a59-863c-8d5df8fdefb2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:29.915636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793b7b-55dd-4a59-863c-8d5df8fdefb2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:29.991515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca458a-8f7d-4e0e-b3fd-e45da870bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:29.992566 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca458a-8f7d-4e0e-b3fd-e45da870bed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:29.992882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca458a-8f7d-4e0e-b3fd-e45da870bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:29.993165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca458a-8f7d-4e0e-b3fd-e45da870bed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:29.994346 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca458a-8f7d-4e0e-b3fd-e45da870bed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:29.995976 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca458a-8f7d-4e0e-b3fd-e45da870bed2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:30.008210 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1332/2652] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:30.012060 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34182c89-6080-43dd-9e26-f86b5012594c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:30.023655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34182c89-6080-43dd-9e26-f86b5012594c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:30.032791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:30.033922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:30.034212 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:30.034496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] PATH_INFO: `/v3/projects/fc6af8a2f6a749c199fcfb659b3cdd33/users/3b47e1d2ec2d47d5a8819869b432f228/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:30.036620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6af8a2f6a749c199fcfb659b3cdd33, user_id=3b47e1d2ec2d47d5a8819869b432f228, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:30.039414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:30.050494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1321/2653] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:30 2022] PUT /identity/v3/projects/fc6af8a2f6a749c199fcfb659b3cdd33/users/3b47e1d2ec2d47d5a8819869b432f228/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:30.118446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc40cba-fa20-4f40-9af1-ce11b50d89e0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:30.118872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc40cba-fa20-4f40-9af1-ce11b50d89e0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:30.119255 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc40cba-fa20-4f40-9af1-ce11b50d89e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:30.153865 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4bc40cba-fa20-4f40-9af1-ce11b50d89e0 None None] MFA Rules not processed for user `3b47e1d2ec2d47d5a8819869b432f228`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:30.159621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4bc40cba-fa20-4f40-9af1-ce11b50d89e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:30.271144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:30.288723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1333/2654] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:30.310728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:30.312815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:30.513473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09569236-4e92-41b4-88d7-56e6a96f99f3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:30.526370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b1e3-244f-420d-bfa2-ebf5546cedf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:30.527058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b1e3-244f-420d-bfa2-ebf5546cedf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:30.527346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b1e3-244f-420d-bfa2-ebf5546cedf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:30.527660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b1e3-244f-420d-bfa2-ebf5546cedf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:30.551848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273b1e3-244f-420d-bfa2-ebf5546cedf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:30.559870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273b1e3-244f-420d-bfa2-ebf5546cedf3 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:30.574304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1322/2655] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:30.635652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc40cba-fa20-4f40-9af1-ce11b50d89e0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:30.637507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4bc40cba-fa20-4f40-9af1-ce11b50d89e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:30.658794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc40cba-fa20-4f40-9af1-ce11b50d89e0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:30.668999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:30.669948 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:30.670246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:30.670661 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:30.766245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:30.769050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:30.800962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1334/2656] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:33.852761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b1e3-244f-420d-bfa2-ebf5546cedf3 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:33.905213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:33.906644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:33.936501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:33.942403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b1e3-244f-420d-bfa2-ebf5546cedf3 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:33.948734 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:33.949628 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:33.949908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:33.950183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] PATH_INFO: `/v3/users/fc1b02d5172c4f58a119f3df87c88e78` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:33.951400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1b02d5172c4f58a119f3df87c88e78)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:33.953379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:33.969494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3d4a5-f01b-436a-90cd-cca593393301 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:33.975368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd46d7-08ee-405b-ab9e-d69f9d8029e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:33.976122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd46d7-08ee-405b-ab9e-d69f9d8029e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:33.976510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd46d7-08ee-405b-ab9e-d69f9d8029e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:33.976848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd46d7-08ee-405b-ab9e-d69f9d8029e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:33.979008 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-91dd46d7-08ee-405b-ab9e-d69f9d8029e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:33.993678 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd46d7-08ee-405b-ab9e-d69f9d8029e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:33.995284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd46d7-08ee-405b-ab9e-d69f9d8029e6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:34.005840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1335/2657] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:11:34.010262 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1b02d5172c4f58a119f3df87c88e78'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.026311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1b02d5172c4f58a119f3df87c88e78'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.027024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1b02d5172c4f58a119f3df87c88e78'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.034591 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1b02d5172c4f58a119f3df87c88e78'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.035265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1b02d5172c4f58a119f3df87c88e78'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.048481 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1323/2658] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:33 2022] DELETE /identity/v3/users/fc1b02d5172c4f58a119f3df87c88e78 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:34.053348 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6ca6d-5f79-460b-9e53-326c4a2a1f29 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:34.053767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6ca6d-5f79-460b-9e53-326c4a2a1f29 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:34.054108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6ca6d-5f79-460b-9e53-326c4a2a1f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:34.073733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-52b6ca6d-5f79-460b-9e53-326c4a2a1f29 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:34.074907 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-52b6ca6d-5f79-460b-9e53-326c4a2a1f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:34.142436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1336/2659] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:11:34.153410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:34.155062 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:34.223485 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff04348-8d8b-4850-92db-136355f961a8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:34.228775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d957a-b04b-4d2a-abb7-4753ec1d896e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:34.229475 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d957a-b04b-4d2a-abb7-4753ec1d896e service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:34.229765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d957a-b04b-4d2a-abb7-4753ec1d896e service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:34.230049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d957a-b04b-4d2a-abb7-4753ec1d896e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:34.241673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d957a-b04b-4d2a-abb7-4753ec1d896e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:34.243855 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d957a-b04b-4d2a-abb7-4753ec1d896e service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:34.257217 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1324/2660] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:34.553828 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6ca6d-5f79-460b-9e53-326c4a2a1f29 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:34.555350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-52b6ca6d-5f79-460b-9e53-326c4a2a1f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:34.571211 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6ca6d-5f79-460b-9e53-326c4a2a1f29 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:34.583114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:34.584362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:34.584973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:34.585374 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] PATH_INFO: `/v3/projects/6cfdf2f2d56841dc84e57a169c6e7d0e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:34.587148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfdf2f2d56841dc84e57a169c6e7d0e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:34.595582 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:34.649947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:34.666567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfdf2f2d56841dc84e57a169c6e7d0e', 'request_id': 'req-bba6b4bd-372c-4620-a8e0-b0865232f579'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.676427 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfdf2f2d56841dc84e57a169c6e7d0e', 'request_id': 'req-bba6b4bd-372c-4620-a8e0-b0865232f579'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.687115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfdf2f2d56841dc84e57a169c6e7d0e', 'request_id': 'req-bba6b4bd-372c-4620-a8e0-b0865232f579'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.706439 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] The token cache is being invalidate because project 6cfdf2f2d56841dc84e57a169c6e7d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:34.706783 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.707712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.710781 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1337/2661] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:34 2022] DELETE /identity/v3/projects/6cfdf2f2d56841dc84e57a169c6e7d0e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:34.715648 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d957a-b04b-4d2a-abb7-4753ec1d896e service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:34.731346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d957a-b04b-4d2a-abb7-4753ec1d896e service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:34.807752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:34.808623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:34.808906 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:34.809181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] PATH_INFO: `/v3/users/8b3b5792efc14a0e9f1f832c5e152ec3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:34.810468 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3b5792efc14a0e9f1f832c5e152ec3)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:34.812699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:34.841542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:34.878782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3b5792efc14a0e9f1f832c5e152ec3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.905948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3b5792efc14a0e9f1f832c5e152ec3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.906953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3b5792efc14a0e9f1f832c5e152ec3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.916805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3b5792efc14a0e9f1f832c5e152ec3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.918630 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3b5792efc14a0e9f1f832c5e152ec3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:34.937646 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1325/2662] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:34 2022] DELETE /identity/v3/users/8b3b5792efc14a0e9f1f832c5e152ec3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:35.505749 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:35.507853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:35.526744 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6b4bd-372c-4620-a8e0-b0865232f579 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:35.640146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:35.641108 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:35.641395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:35.641673 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] PATH_INFO: `/v3/projects/6e2e6231cf4c4be9808995203d7bc5a5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:35.643787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2e6231cf4c4be9808995203d7bc5a5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:35.646815 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:35.714892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2e6231cf4c4be9808995203d7bc5a5', 'request_id': 'req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.725002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2e6231cf4c4be9808995203d7bc5a5', 'request_id': 'req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.736404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2e6231cf4c4be9808995203d7bc5a5', 'request_id': 'req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.758973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] The token cache is being invalidate because project 6e2e6231cf4c4be9808995203d7bc5a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:35.759318 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.760102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.762006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1338/2663] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:35 2022] DELETE /identity/v3/projects/6e2e6231cf4c4be9808995203d7bc5a5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:35.765704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:35.767166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:35.783238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ecd5-ed84-4348-87be-4e8af7f16a60 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:35.887559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:35.888798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:35.890676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:35.890988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] PATH_INFO: `/v3/users/3b47e1d2ec2d47d5a8819869b432f228` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:35.893120 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b47e1d2ec2d47d5a8819869b432f228)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:35.897977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:35.956797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b47e1d2ec2d47d5a8819869b432f228'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.970442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b47e1d2ec2d47d5a8819869b432f228'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.971385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b47e1d2ec2d47d5a8819869b432f228'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.979197 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b47e1d2ec2d47d5a8819869b432f228'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.980057 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-87c0475e-4878-46f1-a896-c7180c5f63e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b47e1d2ec2d47d5a8819869b432f228'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:35.995442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1326/2664] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:35 2022] DELETE /identity/v3/users/3b47e1d2ec2d47d5a8819869b432f228 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:36.355957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:36.358380 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:36.373992 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f85f9-1c4d-4cf4-84d1-b7928a2a9210 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:36.464776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:36.482481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:36.483631 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:36.484112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:36.485311 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] PATH_INFO: `/v3/projects/fc6af8a2f6a749c199fcfb659b3cdd33` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:36.486970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc6af8a2f6a749c199fcfb659b3cdd33)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:36.489580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:36.557626 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc6af8a2f6a749c199fcfb659b3cdd33', 'request_id': 'req-772fea76-1dcb-4e08-9867-c633d76cd777'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:36.567852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc6af8a2f6a749c199fcfb659b3cdd33', 'request_id': 'req-772fea76-1dcb-4e08-9867-c633d76cd777'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:36.576777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc6af8a2f6a749c199fcfb659b3cdd33', 'request_id': 'req-772fea76-1dcb-4e08-9867-c633d76cd777'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:36.599918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] The token cache is being invalidate because project fc6af8a2f6a749c199fcfb659b3cdd33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:36.600234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:36.601986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-772fea76-1dcb-4e08-9867-c633d76cd777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:36.607520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1339/2665] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:36 2022] DELETE /identity/v3/projects/fc6af8a2f6a749c199fcfb659b3cdd33 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:36.617915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c8a2f-aab5-44bf-a525-e8467d7ca217 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:36.617915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c8a2f-aab5-44bf-a525-e8467d7ca217 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:36.618779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c8a2f-aab5-44bf-a525-e8467d7ca217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:36.661533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-094c8a2f-aab5-44bf-a525-e8467d7ca217 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:36.662716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-094c8a2f-aab5-44bf-a525-e8467d7ca217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:36.676898 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:36.761979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1327/2666] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:11:36.764434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67228167-506d-4d12-a028-9a4f8d0cb8f8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:36.765072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67228167-506d-4d12-a028-9a4f8d0cb8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:36.765517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67228167-506d-4d12-a028-9a4f8d0cb8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:36.791497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-67228167-506d-4d12-a028-9a4f8d0cb8f8 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:36.793361 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-67228167-506d-4d12-a028-9a4f8d0cb8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:36.809211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c8a2f-aab5-44bf-a525-e8467d7ca217 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:36.810244 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1340/2667] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:36.822336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c8a2f-aab5-44bf-a525-e8467d7ca217 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:36.829013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:36.830874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:36.831409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:36.832376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:36.833364 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:36.833887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:36.837563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:36.855836 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1328/2668] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:36.860004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67228167-506d-4d12-a028-9a4f8d0cb8f8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:36.880100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67228167-506d-4d12-a028-9a4f8d0cb8f8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:36.888397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de55c38-d77f-4ff9-a3c4-5bbd0dd922b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:36.889176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de55c38-d77f-4ff9-a3c4-5bbd0dd922b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:36.889382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de55c38-d77f-4ff9-a3c4-5bbd0dd922b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:36.889647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de55c38-d77f-4ff9-a3c4-5bbd0dd922b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:36.890423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de55c38-d77f-4ff9-a3c4-5bbd0dd922b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:36.892516 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de55c38-d77f-4ff9-a3c4-5bbd0dd922b4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:36.954620 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1341/2669] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:36 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:36.959583 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:36.973417 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a47a12-dc20-4fdb-8bd8-966dde8b5670 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:37.125112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bbc25-b4d2-48ed-90d3-da1e3d53c2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:37.126204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bbc25-b4d2-48ed-90d3-da1e3d53c2af admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:37.128085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bbc25-b4d2-48ed-90d3-da1e3d53c2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:37.128928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bbc25-b4d2-48ed-90d3-da1e3d53c2af admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:37.129850 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bbc25-b4d2-48ed-90d3-da1e3d53c2af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:37.133655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bbc25-b4d2-48ed-90d3-da1e3d53c2af admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:37.161124 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1329/2670] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:36 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:37.165842 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de55c38-d77f-4ff9-a3c4-5bbd0dd922b4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:37.178909 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de55c38-d77f-4ff9-a3c4-5bbd0dd922b4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:37.183975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942ac44-dfc7-4a05-bb71-aa0c3932cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:37.184708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ac44-dfc7-4a05-bb71-aa0c3932cff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:37.184995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ac44-dfc7-4a05-bb71-aa0c3932cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:37.185276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942ac44-dfc7-4a05-bb71-aa0c3932cff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:37.186022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942ac44-dfc7-4a05-bb71-aa0c3932cff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:37.187996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6942ac44-dfc7-4a05-bb71-aa0c3932cff2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:37.208793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1342/2671] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:37.214908 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bbc25-b4d2-48ed-90d3-da1e3d53c2af admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:37.231454 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bbc25-b4d2-48ed-90d3-da1e3d53c2af admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:37.239910 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951af0e-5244-452f-bcb6-a5f662ca2687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:37.241710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951af0e-5244-452f-bcb6-a5f662ca2687 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:37.241921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951af0e-5244-452f-bcb6-a5f662ca2687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:37.242211 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951af0e-5244-452f-bcb6-a5f662ca2687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:37.242988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951af0e-5244-452f-bcb6-a5f662ca2687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:37.244814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951af0e-5244-452f-bcb6-a5f662ca2687 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:37.263941 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1330/2672] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:37.265477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942ac44-dfc7-4a05-bb71-aa0c3932cff2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:37.278652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942ac44-dfc7-4a05-bb71-aa0c3932cff2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:37.283581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:37.285048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:37.285344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:37.285615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] PATH_INFO: `/v3/projects/ec20f75806154fa3a7d4bd0ea1bb6fee/users/5906262425a246ee8831cd8e137f0c69/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:37.303129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec20f75806154fa3a7d4bd0ea1bb6fee, user_id=5906262425a246ee8831cd8e137f0c69, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:37.306285 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:37.325818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1343/2673] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:37 2022] PUT /identity/v3/projects/ec20f75806154fa3a7d4bd0ea1bb6fee/users/5906262425a246ee8831cd8e137f0c69/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:37.339446 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8775-e07d-4ad4-9cbf-9778a1a02472 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:37.339747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8775-e07d-4ad4-9cbf-9778a1a02472 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:37.340040 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8775-e07d-4ad4-9cbf-9778a1a02472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:37.381771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fb2f8775-e07d-4ad4-9cbf-9778a1a02472 None None] MFA Rules not processed for user `5906262425a246ee8831cd8e137f0c69`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:37.390373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fb2f8775-e07d-4ad4-9cbf-9778a1a02472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:37.464543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1331/2674] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:37.477138 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:37.478841 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:37.545602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea78eae-712e-4251-9013-240574cc5f61 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:37.554248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d1f1d-4bcb-47c1-9f9c-699b8519827c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:37.555127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d1f1d-4bcb-47c1-9f9c-699b8519827c service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:37.555610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d1f1d-4bcb-47c1-9f9c-699b8519827c service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:37.555878 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d1f1d-4bcb-47c1-9f9c-699b8519827c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:37.566676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d1f1d-4bcb-47c1-9f9c-699b8519827c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:37.568144 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d1f1d-4bcb-47c1-9f9c-699b8519827c service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:37.577780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1344/2675] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:38.508378 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8775-e07d-4ad4-9cbf-9778a1a02472 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:38.510458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fb2f8775-e07d-4ad4-9cbf-9778a1a02472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:38.526976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:38.598966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8775-e07d-4ad4-9cbf-9778a1a02472 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:38.604081 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:38.605013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:38.605306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:38.605584 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] PATH_INFO: `/v3/users/bc47c5bf77894aefa6ecde115ed9f88f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:38.606868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc47c5bf77894aefa6ecde115ed9f88f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:38.608831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:38.661676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc47c5bf77894aefa6ecde115ed9f88f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:38.675214 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc47c5bf77894aefa6ecde115ed9f88f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:38.675979 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc47c5bf77894aefa6ecde115ed9f88f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:38.683652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc47c5bf77894aefa6ecde115ed9f88f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:38.684308 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4d1111f8-ed52-40a7-9d76-8071e1a884c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc47c5bf77894aefa6ecde115ed9f88f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:38.700342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1332/2676] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:38 2022] DELETE /identity/v3/users/bc47c5bf77894aefa6ecde115ed9f88f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:38.907267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d1f1d-4bcb-47c1-9f9c-699b8519827c service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:38.919739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d1f1d-4bcb-47c1-9f9c-699b8519827c service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:38.948860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:38.992029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:38.992953 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:38.993241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:38.993519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] PATH_INFO: `/v3/projects/8e98cc41e8064c90b0e45b32e2d78493` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:38.994777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e98cc41e8064c90b0e45b32e2d78493)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:38.996762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.053558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e98cc41e8064c90b0e45b32e2d78493', 'request_id': 'req-f7b48466-6dce-420d-b880-237d0eb306d9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:39.062056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e98cc41e8064c90b0e45b32e2d78493', 'request_id': 'req-f7b48466-6dce-420d-b880-237d0eb306d9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:39.074542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e98cc41e8064c90b0e45b32e2d78493', 'request_id': 'req-f7b48466-6dce-420d-b880-237d0eb306d9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:39.087971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] The token cache is being invalidate because project 8e98cc41e8064c90b0e45b32e2d78493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:39.088342 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:39.089176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f7b48466-6dce-420d-b880-237d0eb306d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:39.091040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1345/2677] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:38 2022] DELETE /identity/v3/projects/8e98cc41e8064c90b0e45b32e2d78493 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:39.096971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd75aa2-4604-46af-a87b-4babc5e20cd7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.097265 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd75aa2-4604-46af-a87b-4babc5e20cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.097719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd75aa2-4604-46af-a87b-4babc5e20cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.118261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bfd75aa2-4604-46af-a87b-4babc5e20cd7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:39.119512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bfd75aa2-4604-46af-a87b-4babc5e20cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:39.199757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1333/2678] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:39.209132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68aac9-0e5f-4492-894e-d0c3c1499e51 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.209482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68aac9-0e5f-4492-894e-d0c3c1499e51 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.209798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68aac9-0e5f-4492-894e-d0c3c1499e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.237007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-db68aac9-0e5f-4492-894e-d0c3c1499e51 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:39.238933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db68aac9-0e5f-4492-894e-d0c3c1499e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:39.249104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1346/2679] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:39.254012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd75aa2-4604-46af-a87b-4babc5e20cd7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.265506 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd75aa2-4604-46af-a87b-4babc5e20cd7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.271499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.272234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.272534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.272843 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.273581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:39.273963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.275774 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.292754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1334/2680] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:39.293656 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68aac9-0e5f-4492-894e-d0c3c1499e51 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.306038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68aac9-0e5f-4492-894e-d0c3c1499e51 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.312368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6dce1-d224-4e73-843a-fd8eca2eb61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.312817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6dce1-d224-4e73-843a-fd8eca2eb61d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.313079 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6dce1-d224-4e73-843a-fd8eca2eb61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.313474 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6dce1-d224-4e73-843a-fd8eca2eb61d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.314097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6dce1-d224-4e73-843a-fd8eca2eb61d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.316806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6dce1-d224-4e73-843a-fd8eca2eb61d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.351074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1347/2681] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:39 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:11:39.355319 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.366756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d018055-b584-411f-954a-3e4e333ec64c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.379725 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:39.427768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3880-1c15-468f-adeb-6a41696613cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.428397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3880-1c15-468f-adeb-6a41696613cb admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.428688 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3880-1c15-468f-adeb-6a41696613cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.428972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3880-1c15-468f-adeb-6a41696613cb admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.429676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a3880-1c15-468f-adeb-6a41696613cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.431624 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a3880-1c15-468f-adeb-6a41696613cb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.451296 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1335/2682] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:39 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:39.455435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6dce1-d224-4e73-843a-fd8eca2eb61d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.466078 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6dce1-d224-4e73-843a-fd8eca2eb61d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.470636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938416a-18ef-4537-a1df-17c75a0c8710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.472176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938416a-18ef-4537-a1df-17c75a0c8710 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.472502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938416a-18ef-4537-a1df-17c75a0c8710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.472779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938416a-18ef-4537-a1df-17c75a0c8710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.473544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938416a-18ef-4537-a1df-17c75a0c8710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.475489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938416a-18ef-4537-a1df-17c75a0c8710 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.485780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1348/2683] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:39.490464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3880-1c15-468f-adeb-6a41696613cb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.500489 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3880-1c15-468f-adeb-6a41696613cb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.505358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6da1b0-6bcd-4749-8522-9aad3cafe333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.506020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6da1b0-6bcd-4749-8522-9aad3cafe333 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.506312 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6da1b0-6bcd-4749-8522-9aad3cafe333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.506595 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6da1b0-6bcd-4749-8522-9aad3cafe333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.507318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6da1b0-6bcd-4749-8522-9aad3cafe333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.508959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6da1b0-6bcd-4749-8522-9aad3cafe333 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.519291 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1336/2684] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:39.523484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938416a-18ef-4537-a1df-17c75a0c8710 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.533980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938416a-18ef-4537-a1df-17c75a0c8710 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.538900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5f26e-cb76-43ad-b05f-09c7a6a827ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.539775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5f26e-cb76-43ad-b05f-09c7a6a827ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.540370 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5f26e-cb76-43ad-b05f-09c7a6a827ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.540647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5f26e-cb76-43ad-b05f-09c7a6a827ba admin admin] PATH_INFO: `/v3/projects/e4c13d31f992483cb0b92827e5fc837c/users/237ea166617c40e08e809f76be5c6019/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.555692 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5f26e-cb76-43ad-b05f-09c7a6a827ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c13d31f992483cb0b92827e5fc837c, user_id=237ea166617c40e08e809f76be5c6019, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.558384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5f26e-cb76-43ad-b05f-09c7a6a827ba admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.571383 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1349/2685] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:39 2022] PUT /identity/v3/projects/e4c13d31f992483cb0b92827e5fc837c/users/237ea166617c40e08e809f76be5c6019/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:39.581618 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96ec6-af3f-42ec-b304-8b80d32faf2e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.581918 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96ec6-af3f-42ec-b304-8b80d32faf2e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.582222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96ec6-af3f-42ec-b304-8b80d32faf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.611201 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-42f96ec6-af3f-42ec-b304-8b80d32faf2e None None] MFA Rules not processed for user `237ea166617c40e08e809f76be5c6019`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:39.613077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-42f96ec6-af3f-42ec-b304-8b80d32faf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:39.684817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1337/2686] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:39.690127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5f26e-cb76-43ad-b05f-09c7a6a827ba admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.743203 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:39.756206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5f26e-cb76-43ad-b05f-09c7a6a827ba admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.761627 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a0f18-df56-45b2-9243-ec8e7f71f644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.762290 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a0f18-df56-45b2-9243-ec8e7f71f644 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.762581 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a0f18-df56-45b2-9243-ec8e7f71f644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.762869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a0f18-df56-45b2-9243-ec8e7f71f644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.763564 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a0f18-df56-45b2-9243-ec8e7f71f644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.765522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a0f18-df56-45b2-9243-ec8e7f71f644 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.793762 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1350/2687] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:39 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:39.798066 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f96ec6-af3f-42ec-b304-8b80d32faf2e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.809524 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f96ec6-af3f-42ec-b304-8b80d32faf2e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.869345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b893804-f351-4c1d-9bf9-1f397dac17fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.870037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b893804-f351-4c1d-9bf9-1f397dac17fe admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.870333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b893804-f351-4c1d-9bf9-1f397dac17fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.870629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b893804-f351-4c1d-9bf9-1f397dac17fe admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.871297 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b893804-f351-4c1d-9bf9-1f397dac17fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.873304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b893804-f351-4c1d-9bf9-1f397dac17fe admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.894581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1338/2688] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:39 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:39.898659 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a0f18-df56-45b2-9243-ec8e7f71f644 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.909633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a0f18-df56-45b2-9243-ec8e7f71f644 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.914302 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27ea9e-20e5-43db-bc75-27fec900a6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.914970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27ea9e-20e5-43db-bc75-27fec900a6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.915265 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27ea9e-20e5-43db-bc75-27fec900a6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.915544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27ea9e-20e5-43db-bc75-27fec900a6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.916312 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27ea9e-20e5-43db-bc75-27fec900a6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.917931 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27ea9e-20e5-43db-bc75-27fec900a6d5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.929159 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1351/2689] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:39.933376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b893804-f351-4c1d-9bf9-1f397dac17fe admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.946980 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b893804-f351-4c1d-9bf9-1f397dac17fe admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.951667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe4e05-1aac-456a-a64e-f2184aeb43f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.952369 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4e05-1aac-456a-a64e-f2184aeb43f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.952660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4e05-1aac-456a-a64e-f2184aeb43f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.952934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4e05-1aac-456a-a64e-f2184aeb43f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:39.953657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe4e05-1aac-456a-a64e-f2184aeb43f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:39.955772 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe4e05-1aac-456a-a64e-f2184aeb43f9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:39.966397 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1339/2690] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:39.971483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27ea9e-20e5-43db-bc75-27fec900a6d5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:39.982172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27ea9e-20e5-43db-bc75-27fec900a6d5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:39.987307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0342d-2076-442d-930d-5cfd6be953cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:39.988326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0342d-2076-442d-930d-5cfd6be953cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:39.988608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0342d-2076-442d-930d-5cfd6be953cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:39.988884 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0342d-2076-442d-930d-5cfd6be953cb admin admin] PATH_INFO: `/v3/projects/84d48c867f99428c8341d10cda2511ab/users/5775b5a9dcdf4d7dbc91ae739d2892c9/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.003615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0342d-2076-442d-930d-5cfd6be953cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d48c867f99428c8341d10cda2511ab, user_id=5775b5a9dcdf4d7dbc91ae739d2892c9, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:40.006235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0342d-2076-442d-930d-5cfd6be953cb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:40.019624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1352/2691] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:39 2022] PUT /identity/v3/projects/84d48c867f99428c8341d10cda2511ab/users/5775b5a9dcdf4d7dbc91ae739d2892c9/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:40.030912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8bf0b-1c67-4998-937b-6014b05c0dc0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.031207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8bf0b-1c67-4998-937b-6014b05c0dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.031497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8bf0b-1c67-4998-937b-6014b05c0dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.039146 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:40.059728 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7bc8bf0b-1c67-4998-937b-6014b05c0dc0 None None] MFA Rules not processed for user `5775b5a9dcdf4d7dbc91ae739d2892c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:40.060969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7bc8bf0b-1c67-4998-937b-6014b05c0dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:40.128143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1340/2692] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:40.132807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0342d-2076-442d-930d-5cfd6be953cb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:40.195024 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0342d-2076-442d-930d-5cfd6be953cb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:40.199746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43107680-3665-421b-bbec-2b64962df7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:40.200436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43107680-3665-421b-bbec-2b64962df7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.200725 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43107680-3665-421b-bbec-2b64962df7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.201010 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43107680-3665-421b-bbec-2b64962df7d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.201714 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43107680-3665-421b-bbec-2b64962df7d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:40.203636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43107680-3665-421b-bbec-2b64962df7d3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:40.230176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1353/2693] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:40 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:40.234314 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8bf0b-1c67-4998-937b-6014b05c0dc0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:40.246118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8bf0b-1c67-4998-937b-6014b05c0dc0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:40.315493 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa166ea-f950-4b12-a098-ec1c6a9ade02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:40.316208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa166ea-f950-4b12-a098-ec1c6a9ade02 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.316499 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa166ea-f950-4b12-a098-ec1c6a9ade02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.316781 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa166ea-f950-4b12-a098-ec1c6a9ade02 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.317497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa166ea-f950-4b12-a098-ec1c6a9ade02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:40.319432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa166ea-f950-4b12-a098-ec1c6a9ade02 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:40.342919 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1341/2694] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:40 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:40.347296 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43107680-3665-421b-bbec-2b64962df7d3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:40.363194 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43107680-3665-421b-bbec-2b64962df7d3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:40.368734 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd207-18c3-458d-97d1-c39cfbfa4d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:40.369640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd207-18c3-458d-97d1-c39cfbfa4d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.369920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd207-18c3-458d-97d1-c39cfbfa4d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.370207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd207-18c3-458d-97d1-c39cfbfa4d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.370946 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fd207-18c3-458d-97d1-c39cfbfa4d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:40.372616 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fd207-18c3-458d-97d1-c39cfbfa4d1c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:40.396093 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1354/2695] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:40.397007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa166ea-f950-4b12-a098-ec1c6a9ade02 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:40.412598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa166ea-f950-4b12-a098-ec1c6a9ade02 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:40.417527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4113ba9-b082-4a84-919e-669d4e4c5a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:40.418427 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4113ba9-b082-4a84-919e-669d4e4c5a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.418712 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4113ba9-b082-4a84-919e-669d4e4c5a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.418988 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4113ba9-b082-4a84-919e-669d4e4c5a92 admin admin] PATH_INFO: `/v3/projects/aeb6c5558e8a4a1faefe7691e6281fb7/users/fe4998a3623b4d4f841f645445e376fb/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.435083 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4113ba9-b082-4a84-919e-669d4e4c5a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb6c5558e8a4a1faefe7691e6281fb7, user_id=fe4998a3623b4d4f841f645445e376fb, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:40.437887 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4113ba9-b082-4a84-919e-669d4e4c5a92 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:40.452492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1342/2696] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:40 2022] PUT /identity/v3/projects/aeb6c5558e8a4a1faefe7691e6281fb7/users/fe4998a3623b4d4f841f645445e376fb/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:40.457509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd207-18c3-458d-97d1-c39cfbfa4d1c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:40.467365 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd207-18c3-458d-97d1-c39cfbfa4d1c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:40.524235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21b6e-fece-427f-917e-df25a3f23f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:40.524910 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21b6e-fece-427f-917e-df25a3f23f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.525195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21b6e-fece-427f-917e-df25a3f23f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.525471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21b6e-fece-427f-917e-df25a3f23f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.526502 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d21b6e-fece-427f-917e-df25a3f23f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:40.527819 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d21b6e-fece-427f-917e-df25a3f23f09 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:40.538359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1355/2697] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:40.542735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4113ba9-b082-4a84-919e-669d4e4c5a92 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:40.553515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4113ba9-b082-4a84-919e-669d4e4c5a92 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:40.558263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:40.559165 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.559448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.559815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] PATH_INFO: `/v3/projects/aeb6c5558e8a4a1faefe7691e6281fb7/users/fe4998a3623b4d4f841f645445e376fb/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.562226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb6c5558e8a4a1faefe7691e6281fb7, user_id=fe4998a3623b4d4f841f645445e376fb, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:40.565716 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:40.577179 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1343/2698] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:40 2022] PUT /identity/v3/projects/aeb6c5558e8a4a1faefe7691e6281fb7/users/fe4998a3623b4d4f841f645445e376fb/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:40.588269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef22c7-1ca9-41be-aabc-aff3135c6a23 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.588568 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef22c7-1ca9-41be-aabc-aff3135c6a23 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.588852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef22c7-1ca9-41be-aabc-aff3135c6a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.608510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:40.616655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9aef22c7-1ca9-41be-aabc-aff3135c6a23 None None] MFA Rules not processed for user `fe4998a3623b4d4f841f645445e376fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:40.618202 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9aef22c7-1ca9-41be-aabc-aff3135c6a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:40.685880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1356/2699] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:40.699141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:40.700912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:40.753643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:40.760682 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65216016-089b-49d5-9f4c-9a15d94fd6ad admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:40.764761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e3f5a-4588-45d6-b1ec-9f228fc19ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:40.765559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3f5a-4588-45d6-b1ec-9f228fc19ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:40.765849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3f5a-4588-45d6-b1ec-9f228fc19ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:40.766127 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3f5a-4588-45d6-b1ec-9f228fc19ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:40.776404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3f5a-4588-45d6-b1ec-9f228fc19ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:40.777864 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3f5a-4588-45d6-b1ec-9f228fc19ae5 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:40.788574 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1344/2700] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:41.287847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef22c7-1ca9-41be-aabc-aff3135c6a23 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:41.289359 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9aef22c7-1ca9-41be-aabc-aff3135c6a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:41.301806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef22c7-1ca9-41be-aabc-aff3135c6a23 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:41.305567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:41.306209 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:41.306497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:41.306808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:41.368061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:41.369595 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:41.384238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1357/2701] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:46.154084 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fc7fc-4ea0-46fc-8ba1-784e68561c2b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:46.154636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fc7fc-4ea0-46fc-8ba1-784e68561c2b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:46.154636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fc7fc-4ea0-46fc-8ba1-784e68561c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:46.175606 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d12fc7fc-4ea0-46fc-8ba1-784e68561c2b None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:46.177042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d12fc7fc-4ea0-46fc-8ba1-784e68561c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:46.237111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1345/2702] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:11:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:46.251526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:46.253040 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:46.341033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c31ab-518b-4f11-a19e-7967569e44e1 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:46.347242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e868238-95e4-4ee5-ae25-7e55217d0522 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:46.349191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868238-95e4-4ee5-ae25-7e55217d0522 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:46.349191 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868238-95e4-4ee5-ae25-7e55217d0522 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:46.349843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868238-95e4-4ee5-ae25-7e55217d0522 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:46.360217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e868238-95e4-4ee5-ae25-7e55217d0522 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:46.363363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e868238-95e4-4ee5-ae25-7e55217d0522 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:46.372399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1358/2703] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:11:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:11:46.920126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774af3e-cb0b-4e0e-b68e-b888e4b28677 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:46.920461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774af3e-cb0b-4e0e-b68e-b888e4b28677 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:46.920767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774af3e-cb0b-4e0e-b68e-b888e4b28677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:46.938996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6774af3e-cb0b-4e0e-b68e-b888e4b28677 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:46.940226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6774af3e-cb0b-4e0e-b68e-b888e4b28677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:46.947976 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1346/2704] 10.222.0.32 () {58 vars in 967 bytes} [Mon May 30 12:11:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:46.960518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e868238-95e4-4ee5-ae25-7e55217d0522 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:46.970929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e868238-95e4-4ee5-ae25-7e55217d0522 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:46.976362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1c882-0f0a-4afc-ae8d-86fec5eb7916 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:46.977007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c882-0f0a-4afc-ae8d-86fec5eb7916 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:46.977319 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c882-0f0a-4afc-ae8d-86fec5eb7916 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:46.977598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c882-0f0a-4afc-ae8d-86fec5eb7916 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:46.985965 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1c882-0f0a-4afc-ae8d-86fec5eb7916 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:46.987586 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1c882-0f0a-4afc-ae8d-86fec5eb7916 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:46.996605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1359/2705] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:11:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 30 12:11:47.199714 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d414538-a964-40c2-ae8f-d8d5145d1462 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:47.200210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d414538-a964-40c2-ae8f-d8d5145d1462 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:47.200331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d414538-a964-40c2-ae8f-d8d5145d1462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:47.218617 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5d414538-a964-40c2-ae8f-d8d5145d1462 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:47.219839 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5d414538-a964-40c2-ae8f-d8d5145d1462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:47.228632 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1347/2706] 10.222.0.32 () {58 vars in 1009 bytes} [Mon May 30 12:11:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:47.241304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1c882-0f0a-4afc-ae8d-86fec5eb7916 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:47.252962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1c882-0f0a-4afc-ae8d-86fec5eb7916 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:47.259101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cb8e3-3997-4b54-866a-3b060afab01c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:47.259754 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cb8e3-3997-4b54-866a-3b060afab01c service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:47.260149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cb8e3-3997-4b54-866a-3b060afab01c service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:47.260552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cb8e3-3997-4b54-866a-3b060afab01c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:47.270539 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cb8e3-3997-4b54-866a-3b060afab01c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:47.272606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cb8e3-3997-4b54-866a-3b060afab01c service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:47.276922 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:47.282794 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1360/2707] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:11:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:11:47.545720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d414538-a964-40c2-ae8f-d8d5145d1462 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:47.557227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d414538-a964-40c2-ae8f-d8d5145d1462 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:47.560977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb14dc-96fd-4426-a283-6ab366fd3296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:47.561643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb14dc-96fd-4426-a283-6ab366fd3296 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:47.561916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb14dc-96fd-4426-a283-6ab366fd3296 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:47.562186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb14dc-96fd-4426-a283-6ab366fd3296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:47.634018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddccf4-33cb-4e3b-8893-4c42fff65d79 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:47.634320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddccf4-33cb-4e3b-8893-4c42fff65d79 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:47.634606 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddccf4-33cb-4e3b-8893-4c42fff65d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:47.638333 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb14dc-96fd-4426-a283-6ab366fd3296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:47.639830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb14dc-96fd-4426-a283-6ab366fd3296 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:47.650324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1348/2708] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:47.663489 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2bddccf4-33cb-4e3b-8893-4c42fff65d79 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:47.665027 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2bddccf4-33cb-4e3b-8893-4c42fff65d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:47.693011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1361/2709] 10.222.0.32 () {58 vars in 967 bytes} [Mon May 30 12:11:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:11:47.707643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb14dc-96fd-4426-a283-6ab366fd3296 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:47.722234 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb14dc-96fd-4426-a283-6ab366fd3296 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:47.727565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9fdd0-818b-48d3-942c-dca79598d180 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:47.728259 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9fdd0-818b-48d3-942c-dca79598d180 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:47.728548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9fdd0-818b-48d3-942c-dca79598d180 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:47.728831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9fdd0-818b-48d3-942c-dca79598d180 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:47.737740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9fdd0-818b-48d3-942c-dca79598d180 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:47.739334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9fdd0-818b-48d3-942c-dca79598d180 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:47.748732 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1349/2710] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:11:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 30 12:11:49.327206 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddccf4-33cb-4e3b-8893-4c42fff65d79 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:49.328793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2bddccf4-33cb-4e3b-8893-4c42fff65d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:49.386002 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddccf4-33cb-4e3b-8893-4c42fff65d79 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:49.389835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2d88-aaf6-43a6-a88c-a6cfd9ac5ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:49.390481 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2d88-aaf6-43a6-a88c-a6cfd9ac5ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:49.390765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2d88-aaf6-43a6-a88c-a6cfd9ac5ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:49.391043 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2d88-aaf6-43a6-a88c-a6cfd9ac5ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:49.393082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fc5e2d88-aaf6-43a6-a88c-a6cfd9ac5ac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:49.461190 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e2d88-aaf6-43a6-a88c-a6cfd9ac5ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:49.462650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e2d88-aaf6-43a6-a88c-a6cfd9ac5ac8 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:49.473233 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1362/2711] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:50.007593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9fdd0-818b-48d3-942c-dca79598d180 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:50.082338 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9fdd0-818b-48d3-942c-dca79598d180 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:50.087282 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:50.088224 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:50.088512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:50.088790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] PATH_INFO: `/v3/users/237ea166617c40e08e809f76be5c6019` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:50.089982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=237ea166617c40e08e809f76be5c6019)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:50.091913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:50.098653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:50.150938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '237ea166617c40e08e809f76be5c6019'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.163616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '237ea166617c40e08e809f76be5c6019'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.164408 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '237ea166617c40e08e809f76be5c6019'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.172109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '237ea166617c40e08e809f76be5c6019'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.172796 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '237ea166617c40e08e809f76be5c6019'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.187366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1350/2712] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:50 2022] DELETE /identity/v3/users/237ea166617c40e08e809f76be5c6019 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:50.193323 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783fb86-ebe8-49d5-8f22-90980c57493d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:50.193625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783fb86-ebe8-49d5-8f22-90980c57493d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:50.193912 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783fb86-ebe8-49d5-8f22-90980c57493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:50.210793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b783fb86-ebe8-49d5-8f22-90980c57493d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:50.212016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b783fb86-ebe8-49d5-8f22-90980c57493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:50.281669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1363/2713] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:50.291561 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:50.293061 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:50.293341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783fb86-ebe8-49d5-8f22-90980c57493d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:50.294780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b783fb86-ebe8-49d5-8f22-90980c57493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:50.308263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783fb86-ebe8-49d5-8f22-90980c57493d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:50.316787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:50.317694 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:50.317985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:50.318264 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] PATH_INFO: `/v3/users/f869410b53b14949a0f066e0e4163a45` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:50.319551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f869410b53b14949a0f066e0e4163a45)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:50.321580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:50.361847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f2f55-d52e-4b41-bef5-b1795f6b5cff admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:50.366239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505af6f7-4669-4c6e-bfd9-da479184f70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:50.366900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af6f7-4669-4c6e-bfd9-da479184f70d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:50.367180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af6f7-4669-4c6e-bfd9-da479184f70d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:50.367470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af6f7-4669-4c6e-bfd9-da479184f70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:50.370316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:50.378073 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505af6f7-4669-4c6e-bfd9-da479184f70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:50.379528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505af6f7-4669-4c6e-bfd9-da479184f70d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:50.384074 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f869410b53b14949a0f066e0e4163a45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.391156 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1351/2714] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:50.402461 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f869410b53b14949a0f066e0e4163a45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.403458 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f869410b53b14949a0f066e0e4163a45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.404330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f869410b53b14949a0f066e0e4163a45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.413399 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f869410b53b14949a0f066e0e4163a45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.432243 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1364/2715] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:50 2022] DELETE /identity/v3/users/f869410b53b14949a0f066e0e4163a45 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:50.436972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb350c-c2b3-4d79-a199-da0b18235a95 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:50.437335 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb350c-c2b3-4d79-a199-da0b18235a95 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:50.437681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb350c-c2b3-4d79-a199-da0b18235a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:50.464376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d9eb350c-c2b3-4d79-a199-da0b18235a95 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:50.465661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d9eb350c-c2b3-4d79-a199-da0b18235a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:50.606343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1352/2716] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:50.641325 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:50.644363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:50.726962 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb350c-c2b3-4d79-a199-da0b18235a95 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:50.729978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d9eb350c-c2b3-4d79-a199-da0b18235a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:50.760752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb350c-c2b3-4d79-a199-da0b18235a95 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:50.762166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b70f7-0f40-4557-8e71-f5cac1b04703 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:50.766769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:50.769484 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:50.770049 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:50.770559 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] PATH_INFO: `/v3/projects/e4c13d31f992483cb0b92827e5fc837c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:50.772257 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4c13d31f992483cb0b92827e5fc837c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:50.772856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65b6c3-e349-4497-9090-79f57bf11dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:50.775179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b6c3-e349-4497-9090-79f57bf11dce service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:50.775496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b6c3-e349-4497-9090-79f57bf11dce service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:50.775785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65b6c3-e349-4497-9090-79f57bf11dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:50.776649 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:50.797532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65b6c3-e349-4497-9090-79f57bf11dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:50.804535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65b6c3-e349-4497-9090-79f57bf11dce service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:50.830390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1365/2717] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:50.838516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:50.881210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4c13d31f992483cb0b92827e5fc837c', 'request_id': 'req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.888226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4c13d31f992483cb0b92827e5fc837c', 'request_id': 'req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.894912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4c13d31f992483cb0b92827e5fc837c', 'request_id': 'req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.909087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] The token cache is being invalidate because project e4c13d31f992483cb0b92827e5fc837c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:50.909448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.910386 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:50.912011 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1353/2718] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:50 2022] DELETE /identity/v3/projects/e4c13d31f992483cb0b92827e5fc837c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:50.916026 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65b6c3-e349-4497-9090-79f57bf11dce service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:50.927118 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65b6c3-e349-4497-9090-79f57bf11dce service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.006440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.007330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.007920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.007920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] PATH_INFO: `/v3/users/5775b5a9dcdf4d7dbc91ae739d2892c9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.009155 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5775b5a9dcdf4d7dbc91ae739d2892c9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.011276 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.072532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.075612 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:51.076962 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5775b5a9dcdf4d7dbc91ae739d2892c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.089740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:51.091430 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4862a-3e92-4e92-bc09-77a5d01e70d6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.100970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5775b5a9dcdf4d7dbc91ae739d2892c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.101723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5775b5a9dcdf4d7dbc91ae739d2892c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.102467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5775b5a9dcdf4d7dbc91ae739d2892c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.105715 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.107190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.107692 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.108031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] PATH_INFO: `/v3/projects/0b9ea0feaee34c639f228a4f320f1aa3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.109699 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9ea0feaee34c639f228a4f320f1aa3)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.111782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.114723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea87e-b501-4ee6-88a5-4080d2ad9545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5775b5a9dcdf4d7dbc91ae739d2892c9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.129770 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1366/2719] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:50 2022] DELETE /identity/v3/users/5775b5a9dcdf4d7dbc91ae739d2892c9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:51.179144 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9ea0feaee34c639f228a4f320f1aa3', 'request_id': 'req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.186410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9ea0feaee34c639f228a4f320f1aa3', 'request_id': 'req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.193310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9ea0feaee34c639f228a4f320f1aa3', 'request_id': 'req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.211719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] The token cache is being invalidate because project 0b9ea0feaee34c639f228a4f320f1aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:51.212038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.212757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4e4fa1ef-62b6-4928-9bdb-0be7476f5475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.214375 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1354/2720] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:51 2022] DELETE /identity/v3/projects/0b9ea0feaee34c639f228a4f320f1aa3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:51.219377 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9c068-483d-4a43-8bbe-c56c0f56d55a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.219667 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9c068-483d-4a43-8bbe-c56c0f56d55a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.220018 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9c068-483d-4a43-8bbe-c56c0f56d55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.240426 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d3f9c068-483d-4a43-8bbe-c56c0f56d55a None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:51.241782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d3f9c068-483d-4a43-8bbe-c56c0f56d55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:51.313718 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1367/2721] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:51.322155 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f32fba-8684-4b2d-8a20-b46f1d82bec7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.322373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f32fba-8684-4b2d-8a20-b46f1d82bec7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.322653 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f32fba-8684-4b2d-8a20-b46f1d82bec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.342126 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-38f32fba-8684-4b2d-8a20-b46f1d82bec7 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:51.343339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-38f32fba-8684-4b2d-8a20-b46f1d82bec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:51.351824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1355/2722] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:11:51.356758 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9c068-483d-4a43-8bbe-c56c0f56d55a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.367366 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9c068-483d-4a43-8bbe-c56c0f56d55a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.372710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.373371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.373729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.374081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.374985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:51.375330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.377284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.401779 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1368/2723] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:51.403085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f32fba-8684-4b2d-8a20-b46f1d82bec7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.412996 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f32fba-8684-4b2d-8a20-b46f1d82bec7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.417815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.418737 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.418876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.419768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.419959 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.422247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.472001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1356/2724] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:51 2022] POST /identity/v3/projects => generated 397 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:51.476847 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.488619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477bc20-10c6-4451-9388-4d89050e452a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.544509 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.546439 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:51.621836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b0647-f5aa-4ca2-b402-8832acb6bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.622767 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0647-f5aa-4ca2-b402-8832acb6bb9e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.623200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0647-f5aa-4ca2-b402-8832acb6bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.623493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0647-f5aa-4ca2-b402-8832acb6bb9e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.624506 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b0647-f5aa-4ca2-b402-8832acb6bb9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.628146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b0647-f5aa-4ca2-b402-8832acb6bb9e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.661193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1369/2725] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:51 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:51.662454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b0647-f5aa-4ca2-b402-8832acb6bb9e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.672041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada637f-9771-4e26-85c6-560d711bb9f6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.679434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b0647-f5aa-4ca2-b402-8832acb6bb9e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.679585 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.680657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.681012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.681643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] PATH_INFO: `/v3/projects/84d48c867f99428c8341d10cda2511ab` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.682927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84d48c867f99428c8341d10cda2511ab)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.685982 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.688967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1d359-48e3-42f4-b1fa-88a8490f903a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.689839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1d359-48e3-42f4-b1fa-88a8490f903a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.690164 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1d359-48e3-42f4-b1fa-88a8490f903a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.690511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1d359-48e3-42f4-b1fa-88a8490f903a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.691381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1d359-48e3-42f4-b1fa-88a8490f903a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.693435 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1d359-48e3-42f4-b1fa-88a8490f903a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.698555 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:51.708720 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1370/2726] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:51.713641 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1d359-48e3-42f4-b1fa-88a8490f903a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.727812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1d359-48e3-42f4-b1fa-88a8490f903a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.735590 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23115f8-fec9-4721-a53f-39718b078b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.736332 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23115f8-fec9-4721-a53f-39718b078b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.737689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23115f8-fec9-4721-a53f-39718b078b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.738281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23115f8-fec9-4721-a53f-39718b078b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.739936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23115f8-fec9-4721-a53f-39718b078b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.742902 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23115f8-fec9-4721-a53f-39718b078b0f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.757822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1371/2727] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:51.758656 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84d48c867f99428c8341d10cda2511ab', 'request_id': 'req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.762215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23115f8-fec9-4721-a53f-39718b078b0f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.767142 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84d48c867f99428c8341d10cda2511ab', 'request_id': 'req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.776011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23115f8-fec9-4721-a53f-39718b078b0f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.776292 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84d48c867f99428c8341d10cda2511ab', 'request_id': 'req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.802355 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] The token cache is being invalidate because project 84d48c867f99428c8341d10cda2511ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:51.802651 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.803342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:51.805866 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1357/2728] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:51 2022] DELETE /identity/v3/projects/84d48c867f99428c8341d10cda2511ab => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:51.809706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:51.811443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:51.829163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:51.884630 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfab48f-a6fd-48cd-82aa-41a12cab1f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:51.885825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfab48f-a6fd-48cd-82aa-41a12cab1f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.886127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfab48f-a6fd-48cd-82aa-41a12cab1f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.886406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfab48f-a6fd-48cd-82aa-41a12cab1f15 admin admin] PATH_INFO: `/v3/projects/0d5f8209ebbd42a4897c2a77a3e80cda/users/4a3e23ac329048999100946fe57e6e6d/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.909381 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfab48f-a6fd-48cd-82aa-41a12cab1f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5f8209ebbd42a4897c2a77a3e80cda, user_id=4a3e23ac329048999100946fe57e6e6d, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:51.913999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfab48f-a6fd-48cd-82aa-41a12cab1f15 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:51.935005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1372/2729] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:51 2022] PUT /identity/v3/projects/0d5f8209ebbd42a4897c2a77a3e80cda/users/4a3e23ac329048999100946fe57e6e6d/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:51.944912 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0377a-6b6e-42ad-a100-8f6fdfaaa7f7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:51.946937 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e020e-3bcd-4444-9460-e688c3104189 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:51.947351 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e020e-3bcd-4444-9460-e688c3104189 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:51.947607 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e020e-3bcd-4444-9460-e688c3104189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:51.981614 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2a5e020e-3bcd-4444-9460-e688c3104189 None None] MFA Rules not processed for user `4a3e23ac329048999100946fe57e6e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:51.982945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2a5e020e-3bcd-4444-9460-e688c3104189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:52.031883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:52.032827 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:52.033115 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:52.033392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] PATH_INFO: `/v3/users/fe4998a3623b4d4f841f645445e376fb` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:52.036200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe4998a3623b4d4f841f645445e376fb)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:52.037511 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:52.082792 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1373/2730] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:11:52.091913 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e020e-3bcd-4444-9460-e688c3104189 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:52.094088 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2a5e020e-3bcd-4444-9460-e688c3104189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:52.106609 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:52.108456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe4998a3623b4d4f841f645445e376fb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.129794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe4998a3623b4d4f841f645445e376fb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.131385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe4998a3623b4d4f841f645445e376fb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.140635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe4998a3623b4d4f841f645445e376fb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.141706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe4998a3623b4d4f841f645445e376fb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.163779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1358/2731] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:51 2022] DELETE /identity/v3/users/fe4998a3623b4d4f841f645445e376fb => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:52.181356 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e020e-3bcd-4444-9460-e688c3104189 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:52.263784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d170d9-7538-4cad-ad6b-e8c51d4fc203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:52.264496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d170d9-7538-4cad-ad6b-e8c51d4fc203 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:52.264787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d170d9-7538-4cad-ad6b-e8c51d4fc203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:52.265132 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d170d9-7538-4cad-ad6b-e8c51d4fc203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:52.267172 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-16d170d9-7538-4cad-ad6b-e8c51d4fc203 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:52.285092 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:52.353222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d170d9-7538-4cad-ad6b-e8c51d4fc203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:52.357868 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d170d9-7538-4cad-ad6b-e8c51d4fc203 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:52.366944 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1374/2732] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:52.545790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:52.547223 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:52.725132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e410aa5-b3e2-4b4f-ab07-0373d8493960 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:52.736471 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:52.737944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:52.738283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:52.738710 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] PATH_INFO: `/v3/projects/aeb6c5558e8a4a1faefe7691e6281fb7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:52.740473 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb6c5558e8a4a1faefe7691e6281fb7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:52.742635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:52.829604 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb6c5558e8a4a1faefe7691e6281fb7', 'request_id': 'req-036e3b42-85da-4999-9397-ae89fef4f5dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.838370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb6c5558e8a4a1faefe7691e6281fb7', 'request_id': 'req-036e3b42-85da-4999-9397-ae89fef4f5dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.845771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb6c5558e8a4a1faefe7691e6281fb7', 'request_id': 'req-036e3b42-85da-4999-9397-ae89fef4f5dd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.861562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] The token cache is being invalidate because project aeb6c5558e8a4a1faefe7691e6281fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:52.861983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.863069 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-036e3b42-85da-4999-9397-ae89fef4f5dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:52.865448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1359/2733] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:52 2022] DELETE /identity/v3/projects/aeb6c5558e8a4a1faefe7691e6281fb7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:52.873262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252b3f1-b438-45eb-9550-2706ae8bc2ca None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:52.873845 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252b3f1-b438-45eb-9550-2706ae8bc2ca None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:52.874167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252b3f1-b438-45eb-9550-2706ae8bc2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:52.924406 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2252b3f1-b438-45eb-9550-2706ae8bc2ca None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:52.928638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2252b3f1-b438-45eb-9550-2706ae8bc2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:53.064316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1375/2734] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:53.072939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c39c18-d3b0-4b88-bb3f-ab7398f27e12 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.073255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c39c18-d3b0-4b88-bb3f-ab7398f27e12 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.073522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c39c18-d3b0-4b88-bb3f-ab7398f27e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.103681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-72c39c18-d3b0-4b88-bb3f-ab7398f27e12 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:53.105972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-72c39c18-d3b0-4b88-bb3f-ab7398f27e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:53.134183 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1360/2735] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:53.135288 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252b3f1-b438-45eb-9550-2706ae8bc2ca None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:53.152851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252b3f1-b438-45eb-9550-2706ae8bc2ca None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:53.162514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:53.163514 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.163798 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.164084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.165666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:53.165988 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:53.167635 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:53.188776 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1376/2736] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:53.190423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c39c18-d3b0-4b88-bb3f-ab7398f27e12 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:53.204605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c39c18-d3b0-4b88-bb3f-ab7398f27e12 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:53.211068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832f106-82ef-4478-972f-b9fd3be8ddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:53.212283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f106-82ef-4478-972f-b9fd3be8ddb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.213341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f106-82ef-4478-972f-b9fd3be8ddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.213783 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f106-82ef-4478-972f-b9fd3be8ddb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.216746 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832f106-82ef-4478-972f-b9fd3be8ddb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:53.220798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832f106-82ef-4478-972f-b9fd3be8ddb6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:53.258385 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1361/2737] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:53 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:53.260012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:53.275857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b12ee6-4044-4880-8ce5-8a9f0ac2f218 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:53.357768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99467c-b61b-4310-a466-08b5bf0963eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:53.358834 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99467c-b61b-4310-a466-08b5bf0963eb admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.359133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99467c-b61b-4310-a466-08b5bf0963eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.359475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99467c-b61b-4310-a466-08b5bf0963eb admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.360189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99467c-b61b-4310-a466-08b5bf0963eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:53.362726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99467c-b61b-4310-a466-08b5bf0963eb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:53.388530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1377/2738] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:53 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:53.393733 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832f106-82ef-4478-972f-b9fd3be8ddb6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:53.407343 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832f106-82ef-4478-972f-b9fd3be8ddb6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:53.418334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c67a6-7a1b-4c76-9302-0aa53d940bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:53.419254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c67a6-7a1b-4c76-9302-0aa53d940bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.419553 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c67a6-7a1b-4c76-9302-0aa53d940bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.419841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c67a6-7a1b-4c76-9302-0aa53d940bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.420614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c67a6-7a1b-4c76-9302-0aa53d940bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:53.422422 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c67a6-7a1b-4c76-9302-0aa53d940bc8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:53.442916 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99467c-b61b-4310-a466-08b5bf0963eb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:53.444347 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1362/2739] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:53.455657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99467c-b61b-4310-a466-08b5bf0963eb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:53.467719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c45b5-89a4-4937-9154-85cfe2ae34fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:53.469139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c45b5-89a4-4937-9154-85cfe2ae34fc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.469475 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c45b5-89a4-4937-9154-85cfe2ae34fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.469817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c45b5-89a4-4937-9154-85cfe2ae34fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.470548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c45b5-89a4-4937-9154-85cfe2ae34fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:53.473284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c45b5-89a4-4937-9154-85cfe2ae34fc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:53.489452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1378/2740] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:53.490510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c67a6-7a1b-4c76-9302-0aa53d940bc8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:53.501404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c67a6-7a1b-4c76-9302-0aa53d940bc8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:53.506514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b59c6d-ed4d-41c8-ada9-5accb52cf692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:53.507409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b59c6d-ed4d-41c8-ada9-5accb52cf692 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.507696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b59c6d-ed4d-41c8-ada9-5accb52cf692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.507978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b59c6d-ed4d-41c8-ada9-5accb52cf692 admin admin] PATH_INFO: `/v3/projects/d1ccf47e9da946619d987466943ed772/users/5d6e4a5c471347e5bf4811a7dce7108e/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.521024 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:53.525466 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b59c6d-ed4d-41c8-ada9-5accb52cf692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ccf47e9da946619d987466943ed772, user_id=5d6e4a5c471347e5bf4811a7dce7108e, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:53.528175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b59c6d-ed4d-41c8-ada9-5accb52cf692 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:53.542727 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1363/2741] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:53 2022] PUT /identity/v3/projects/d1ccf47e9da946619d987466943ed772/users/5d6e4a5c471347e5bf4811a7dce7108e/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:11:53.555579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c33c3-82ad-48d5-8579-9fddebebb523 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.555869 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c33c3-82ad-48d5-8579-9fddebebb523 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.556171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c33c3-82ad-48d5-8579-9fddebebb523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.593401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-761c33c3-82ad-48d5-8579-9fddebebb523 None None] MFA Rules not processed for user `5d6e4a5c471347e5bf4811a7dce7108e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:53.595849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-761c33c3-82ad-48d5-8579-9fddebebb523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:53.604334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:53.664999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1379/2742] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:53.669936 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b59c6d-ed4d-41c8-ada9-5accb52cf692 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:53.741771 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b59c6d-ed4d-41c8-ada9-5accb52cf692 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:53.746685 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98caa1-8e66-4c0c-95e2-0e477cdd922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:53.747492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98caa1-8e66-4c0c-95e2-0e477cdd922f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.747790 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98caa1-8e66-4c0c-95e2-0e477cdd922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.748072 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98caa1-8e66-4c0c-95e2-0e477cdd922f admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.748804 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98caa1-8e66-4c0c-95e2-0e477cdd922f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:53.750752 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98caa1-8e66-4c0c-95e2-0e477cdd922f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:53.807186 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1364/2743] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:53 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:53.811900 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c33c3-82ad-48d5-8579-9fddebebb523 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:53.833389 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c33c3-82ad-48d5-8579-9fddebebb523 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:53.961128 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ac02c-2678-4afd-b7b1-47d16c313ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:53.961735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ac02c-2678-4afd-b7b1-47d16c313ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:53.962430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ac02c-2678-4afd-b7b1-47d16c313ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:53.962430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ac02c-2678-4afd-b7b1-47d16c313ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:53.962891 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ac02c-2678-4afd-b7b1-47d16c313ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:53.964936 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ac02c-2678-4afd-b7b1-47d16c313ab1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:53.993594 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1380/2744] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:53 2022] POST /identity/v3/users => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:53.997368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98caa1-8e66-4c0c-95e2-0e477cdd922f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:54.009657 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98caa1-8e66-4c0c-95e2-0e477cdd922f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:54.016149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692edca5-f8ce-44ea-b8f8-37491f0f3058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:54.029162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692edca5-f8ce-44ea-b8f8-37491f0f3058 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:54.029460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692edca5-f8ce-44ea-b8f8-37491f0f3058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:54.031809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692edca5-f8ce-44ea-b8f8-37491f0f3058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:54.033123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692edca5-f8ce-44ea-b8f8-37491f0f3058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:54.036565 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692edca5-f8ce-44ea-b8f8-37491f0f3058 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:54.052054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1365/2745] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:54.057283 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ac02c-2678-4afd-b7b1-47d16c313ab1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:54.069152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ac02c-2678-4afd-b7b1-47d16c313ab1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:54.074316 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435496ba-1279-42aa-8b1b-62c59b7e378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:54.075291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435496ba-1279-42aa-8b1b-62c59b7e378e admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:54.075585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435496ba-1279-42aa-8b1b-62c59b7e378e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:54.075897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435496ba-1279-42aa-8b1b-62c59b7e378e admin admin] PATH_INFO: `/v3/projects/84a9c61f8a834faab2fb8d2a64a7f1f9/users/c857bdd6123d413a9813aae321de958e/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:54.095517 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435496ba-1279-42aa-8b1b-62c59b7e378e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a9c61f8a834faab2fb8d2a64a7f1f9, user_id=c857bdd6123d413a9813aae321de958e, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:54.101324 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435496ba-1279-42aa-8b1b-62c59b7e378e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:54.129833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1381/2746] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:54 2022] PUT /identity/v3/projects/84a9c61f8a834faab2fb8d2a64a7f1f9/users/c857bdd6123d413a9813aae321de958e/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:54.131190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692edca5-f8ce-44ea-b8f8-37491f0f3058 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:54.143841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692edca5-f8ce-44ea-b8f8-37491f0f3058 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:54.235753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2964b1-4a86-488d-88fc-c3f5868fdc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:54.236620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2964b1-4a86-488d-88fc-c3f5868fdc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:54.237619 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2964b1-4a86-488d-88fc-c3f5868fdc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:54.238345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2964b1-4a86-488d-88fc-c3f5868fdc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:54.239104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2964b1-4a86-488d-88fc-c3f5868fdc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:54.241063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2964b1-4a86-488d-88fc-c3f5868fdc03 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:54.255429 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1366/2747] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:11:54.259420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435496ba-1279-42aa-8b1b-62c59b7e378e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:54.273949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435496ba-1279-42aa-8b1b-62c59b7e378e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:54.279866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:54.281445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:54.281896 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:54.282235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] PATH_INFO: `/v3/projects/84a9c61f8a834faab2fb8d2a64a7f1f9/users/c857bdd6123d413a9813aae321de958e/roles/4d21498572b14478a996ac538be5382e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:54.284825 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a9c61f8a834faab2fb8d2a64a7f1f9, user_id=c857bdd6123d413a9813aae321de958e, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:54.288230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:54.305730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1382/2748] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:54 2022] PUT /identity/v3/projects/84a9c61f8a834faab2fb8d2a64a7f1f9/users/c857bdd6123d413a9813aae321de958e/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:54.317681 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ed710-9ff6-4021-bdd4-21e68eaa1bf8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:54.318152 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ed710-9ff6-4021-bdd4-21e68eaa1bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:54.318508 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ed710-9ff6-4021-bdd4-21e68eaa1bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:54.351438 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-281ed710-9ff6-4021-bdd4-21e68eaa1bf8 None None] MFA Rules not processed for user `c857bdd6123d413a9813aae321de958e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:54.353076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-281ed710-9ff6-4021-bdd4-21e68eaa1bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:54.444735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:54.477658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1367/2749] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:54.492254 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:54.494239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:54.579947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93775e88-2447-4e20-b0c0-09bb729f9338 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:54.592796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0416-8617-4942-bcf9-8e1a7f5cbaf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:54.593491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0416-8617-4942-bcf9-8e1a7f5cbaf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:54.593778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0416-8617-4942-bcf9-8e1a7f5cbaf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:54.594061 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0416-8617-4942-bcf9-8e1a7f5cbaf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:54.617991 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e0416-8617-4942-bcf9-8e1a7f5cbaf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:54.620175 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e0416-8617-4942-bcf9-8e1a7f5cbaf4 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:54.634280 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1383/2750] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:11:54.723614 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ed710-9ff6-4021-bdd4-21e68eaa1bf8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:54.734409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ed710-9ff6-4021-bdd4-21e68eaa1bf8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:54.739890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f892ac-5c96-4cb7-af8a-f819c3fca942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:54.740687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f892ac-5c96-4cb7-af8a-f819c3fca942 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:54.741527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f892ac-5c96-4cb7-af8a-f819c3fca942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:54.741527 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f892ac-5c96-4cb7-af8a-f819c3fca942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:54.807992 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f892ac-5c96-4cb7-af8a-f819c3fca942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:54.809472 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f892ac-5c96-4cb7-af8a-f819c3fca942 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:54.819335 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1368/2751] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:55.087522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0416-8617-4942-bcf9-8e1a7f5cbaf4 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:55.093016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:55.152241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0416-8617-4942-bcf9-8e1a7f5cbaf4 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:55.156849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:55.157719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:55.158009 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:55.158291 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] PATH_INFO: `/v3/users/5d6e4a5c471347e5bf4811a7dce7108e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:55.160193 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6e4a5c471347e5bf4811a7dce7108e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:55.162186 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:55.208491 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6e4a5c471347e5bf4811a7dce7108e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.220049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6e4a5c471347e5bf4811a7dce7108e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.220740 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6e4a5c471347e5bf4811a7dce7108e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.221879 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6e4a5c471347e5bf4811a7dce7108e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.228836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6e4a5c471347e5bf4811a7dce7108e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.241911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1384/2752] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:55 2022] DELETE /identity/v3/users/5d6e4a5c471347e5bf4811a7dce7108e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:55.246410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e384b-8491-4c0b-95e9-125dc29053f5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:55.246931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e384b-8491-4c0b-95e9-125dc29053f5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:55.247048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e384b-8491-4c0b-95e9-125dc29053f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:55.263926 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-356e384b-8491-4c0b-95e9-125dc29053f5 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:55.265176 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-356e384b-8491-4c0b-95e9-125dc29053f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:55.326824 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1369/2753] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:55.336643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:55.338147 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:55.400933 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05ba87-719b-423d-9f99-b52ac9b5c16d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:55.404982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d0c3e-5eb4-47c0-9925-8efb203a4519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:55.405636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d0c3e-5eb4-47c0-9925-8efb203a4519 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:55.405926 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d0c3e-5eb4-47c0-9925-8efb203a4519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:55.406207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d0c3e-5eb4-47c0-9925-8efb203a4519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:55.416293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d0c3e-5eb4-47c0-9925-8efb203a4519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:55.417748 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d0c3e-5eb4-47c0-9925-8efb203a4519 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:55.427813 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1385/2754] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:11:55.699603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e384b-8491-4c0b-95e9-125dc29053f5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:55.701056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-356e384b-8491-4c0b-95e9-125dc29053f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:55.714370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e384b-8491-4c0b-95e9-125dc29053f5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:55.719028 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:55.719900 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:55.720253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:55.720917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] PATH_INFO: `/v3/projects/d1ccf47e9da946619d987466943ed772` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:55.722357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ccf47e9da946619d987466943ed772)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:55.724341 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:55.774909 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ccf47e9da946619d987466943ed772', 'request_id': 'req-b27db330-d092-42ca-a47c-05c66994d00d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.781722 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ccf47e9da946619d987466943ed772', 'request_id': 'req-b27db330-d092-42ca-a47c-05c66994d00d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.788258 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ccf47e9da946619d987466943ed772', 'request_id': 'req-b27db330-d092-42ca-a47c-05c66994d00d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.802090 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] The token cache is being invalidate because project d1ccf47e9da946619d987466943ed772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:55.802391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.803037 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.804643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1370/2755] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:55 2022] DELETE /identity/v3/projects/d1ccf47e9da946619d987466943ed772 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:55.808855 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d0c3e-5eb4-47c0-9925-8efb203a4519 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:55.819213 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d0c3e-5eb4-47c0-9925-8efb203a4519 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:55.844336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:55.879115 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:55.879995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:55.880322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:55.880602 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] PATH_INFO: `/v3/users/c857bdd6123d413a9813aae321de958e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:55.881801 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c857bdd6123d413a9813aae321de958e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:55.883750 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:55.931082 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c857bdd6123d413a9813aae321de958e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.943306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c857bdd6123d413a9813aae321de958e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.943970 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c857bdd6123d413a9813aae321de958e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.944836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c857bdd6123d413a9813aae321de958e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.952025 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95774b47-8c5d-4815-8bd5-ae10ff0f5d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c857bdd6123d413a9813aae321de958e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:55.966173 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1386/2756] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:11:55 2022] DELETE /identity/v3/users/c857bdd6123d413a9813aae321de958e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:56.247913 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:56.249437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:56.265442 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27db330-d092-42ca-a47c-05c66994d00d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:56.269779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:56.328516 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:56.329403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.329691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.329975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] PATH_INFO: `/v3/projects/84a9c61f8a834faab2fb8d2a64a7f1f9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.331218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a9c61f8a834faab2fb8d2a64a7f1f9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:56.333175 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:56.382198 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a9c61f8a834faab2fb8d2a64a7f1f9', 'request_id': 'req-b99a5a3c-5314-472c-a84a-455606d31682'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:56.389670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a9c61f8a834faab2fb8d2a64a7f1f9', 'request_id': 'req-b99a5a3c-5314-472c-a84a-455606d31682'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:56.396458 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a9c61f8a834faab2fb8d2a64a7f1f9', 'request_id': 'req-b99a5a3c-5314-472c-a84a-455606d31682'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:56.412740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] The token cache is being invalidate because project 84a9c61f8a834faab2fb8d2a64a7f1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:11:56.413031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:56.413661 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b99a5a3c-5314-472c-a84a-455606d31682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:11:56.415226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1371/2757] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:11:56 2022] DELETE /identity/v3/projects/84a9c61f8a834faab2fb8d2a64a7f1f9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:56.420212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ea195-9b10-4f69-9918-66107e7fa2b9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.420685 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ea195-9b10-4f69-9918-66107e7fa2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.420784 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ea195-9b10-4f69-9918-66107e7fa2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.439207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-450ea195-9b10-4f69-9918-66107e7fa2b9 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:56.441277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-450ea195-9b10-4f69-9918-66107e7fa2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:56.510743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1387/2758] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:56.520367 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cad7-2be5-4952-ab92-5b57764b5eff None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.520662 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cad7-2be5-4952-ab92-5b57764b5eff None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.520944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cad7-2be5-4952-ab92-5b57764b5eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.540349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3bb5cad7-2be5-4952-ab92-5b57764b5eff None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:56.541560 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3bb5cad7-2be5-4952-ab92-5b57764b5eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:56.550189 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1372/2759] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:56.554468 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ea195-9b10-4f69-9918-66107e7fa2b9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:56.565478 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ea195-9b10-4f69-9918-66107e7fa2b9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:56.570127 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:56.570787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.571076 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.576639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.576639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:11:56.576639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:56.576639 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:56.586181 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1388/2760] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:11:56.590426 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5cad7-2be5-4952-ab92-5b57764b5eff None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:56.600665 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5cad7-2be5-4952-ab92-5b57764b5eff None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:56.605381 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76164b2a-5a45-4a26-b3b6-71fefed672ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:56.606041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76164b2a-5a45-4a26-b3b6-71fefed672ad admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.606357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76164b2a-5a45-4a26-b3b6-71fefed672ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.606635 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76164b2a-5a45-4a26-b3b6-71fefed672ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.607321 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76164b2a-5a45-4a26-b3b6-71fefed672ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:56.609261 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76164b2a-5a45-4a26-b3b6-71fefed672ad admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:56.636273 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1373/2761] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:11:56 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:56.640492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:56.650974 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fe7e1-81ed-4024-a70b-47ff8f5f4aa1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:56.681961 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:56.710262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda02aa-9aec-47d6-bbd5-f6d6b1bf3084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:56.710942 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda02aa-9aec-47d6-bbd5-f6d6b1bf3084 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.711230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda02aa-9aec-47d6-bbd5-f6d6b1bf3084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.711512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda02aa-9aec-47d6-bbd5-f6d6b1bf3084 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.712240 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda02aa-9aec-47d6-bbd5-f6d6b1bf3084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:56.714198 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda02aa-9aec-47d6-bbd5-f6d6b1bf3084 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:56.735001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1389/2762] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:11:56 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:11:56.740210 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76164b2a-5a45-4a26-b3b6-71fefed672ad admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:56.751026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76164b2a-5a45-4a26-b3b6-71fefed672ad admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:56.755745 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79918d-0a63-440d-bbf0-918ad1d4b6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:56.756465 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79918d-0a63-440d-bbf0-918ad1d4b6cf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.756756 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79918d-0a63-440d-bbf0-918ad1d4b6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.757051 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79918d-0a63-440d-bbf0-918ad1d4b6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.757776 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79918d-0a63-440d-bbf0-918ad1d4b6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:56.759432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79918d-0a63-440d-bbf0-918ad1d4b6cf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:56.770478 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1374/2763] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:56.774687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda02aa-9aec-47d6-bbd5-f6d6b1bf3084 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:56.785623 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda02aa-9aec-47d6-bbd5-f6d6b1bf3084 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:56.800608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6104f08-e49a-4c8e-ac40-3f9037ac0d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:56.801284 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104f08-e49a-4c8e-ac40-3f9037ac0d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.801592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104f08-e49a-4c8e-ac40-3f9037ac0d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.801853 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6104f08-e49a-4c8e-ac40-3f9037ac0d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.802780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6104f08-e49a-4c8e-ac40-3f9037ac0d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:56.805664 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6104f08-e49a-4c8e-ac40-3f9037ac0d73 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:56.817888 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1390/2764] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:11:56.822772 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79918d-0a63-440d-bbf0-918ad1d4b6cf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:56.838432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79918d-0a63-440d-bbf0-918ad1d4b6cf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:56.844025 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:56.844965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.845252 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.845531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] PATH_INFO: `/v3/projects/9bee19f42aa14e998c676a4b951dd58c/users/94fb4526e77e4c00bf9da389c04d7da2/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.872016 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bee19f42aa14e998c676a4b951dd58c, user_id=94fb4526e77e4c00bf9da389c04d7da2, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:56.874863 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:56.888358 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1375/2765] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:11:56 2022] PUT /identity/v3/projects/9bee19f42aa14e998c676a4b951dd58c/users/94fb4526e77e4c00bf9da389c04d7da2/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:11:56.899747 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861b415-2d81-4f59-a836-330346a8f238 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:56.900153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861b415-2d81-4f59-a836-330346a8f238 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:56.900534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861b415-2d81-4f59-a836-330346a8f238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:56.927532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c861b415-2d81-4f59-a836-330346a8f238 None None] MFA Rules not processed for user `94fb4526e77e4c00bf9da389c04d7da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:11:56.928777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c861b415-2d81-4f59-a836-330346a8f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:56.992644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1391/2766] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:11:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:11:57.003928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:11:57.005570 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:11:57.042404 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:11:57.065937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ac112-3195-4c8e-b6be-2fb393c1636a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:11:57.069760 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae753d-c61b-43f6-a05e-be0d352bb173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:11:57.070410 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae753d-c61b-43f6-a05e-be0d352bb173 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:11:57.070701 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae753d-c61b-43f6-a05e-be0d352bb173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:11:57.070986 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae753d-c61b-43f6-a05e-be0d352bb173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:11:57.079374 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae753d-c61b-43f6-a05e-be0d352bb173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:11:57.081345 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae753d-c61b-43f6-a05e-be0d352bb173 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:11:57.090191 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1376/2767] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:11:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:02.370580 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861b415-2d81-4f59-a836-330346a8f238 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:02.371968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c861b415-2d81-4f59-a836-330346a8f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:02.439207 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861b415-2d81-4f59-a836-330346a8f238 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:02.444575 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:02.445443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:02.445726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:02.446006 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] PATH_INFO: `/v3/users/5906262425a246ee8831cd8e137f0c69` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:02.447195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5906262425a246ee8831cd8e137f0c69)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:02.449154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:02.494195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:02.496911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5906262425a246ee8831cd8e137f0c69'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:02.510215 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5906262425a246ee8831cd8e137f0c69'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:02.510882 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5906262425a246ee8831cd8e137f0c69'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:02.511480 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5906262425a246ee8831cd8e137f0c69'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:02.518632 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5906262425a246ee8831cd8e137f0c69'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:02.533140 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1392/2768] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:12:02 2022] DELETE /identity/v3/users/5906262425a246ee8831cd8e137f0c69 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:02.538287 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea30e-3a8d-4995-8fc0-59c39ef2ba85 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:02.538809 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea30e-3a8d-4995-8fc0-59c39ef2ba85 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:02.538927 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea30e-3a8d-4995-8fc0-59c39ef2ba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:02.557054 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8b3ea30e-3a8d-4995-8fc0-59c39ef2ba85 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:02.558274 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8b3ea30e-3a8d-4995-8fc0-59c39ef2ba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:02.623286 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1377/2769] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:02.632743 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:02.636038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:02.705400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769259b-f672-4f5c-a0d2-521e86bb1c2e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:02.710832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39da3a-d1a2-4cf6-b41f-c8667de8f0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:02.711726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39da3a-d1a2-4cf6-b41f-c8667de8f0dc service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:02.712097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39da3a-d1a2-4cf6-b41f-c8667de8f0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:02.712345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39da3a-d1a2-4cf6-b41f-c8667de8f0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:02.723675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39da3a-d1a2-4cf6-b41f-c8667de8f0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:02.725188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39da3a-d1a2-4cf6-b41f-c8667de8f0dc service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:02.737235 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1393/2770] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:03.092117 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea30e-3a8d-4995-8fc0-59c39ef2ba85 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:03.094114 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8b3ea30e-3a8d-4995-8fc0-59c39ef2ba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:03.117769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea30e-3a8d-4995-8fc0-59c39ef2ba85 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:03.127104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:03.128392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.128684 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.128975 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] PATH_INFO: `/v3/projects/ec20f75806154fa3a7d4bd0ea1bb6fee` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.130721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec20f75806154fa3a7d4bd0ea1bb6fee)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:03.147231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:03.243193 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec20f75806154fa3a7d4bd0ea1bb6fee', 'request_id': 'req-ef99f353-4202-4c72-9175-0a2ea5a58adf'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:03.250286 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec20f75806154fa3a7d4bd0ea1bb6fee', 'request_id': 'req-ef99f353-4202-4c72-9175-0a2ea5a58adf'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:03.258112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec20f75806154fa3a7d4bd0ea1bb6fee', 'request_id': 'req-ef99f353-4202-4c72-9175-0a2ea5a58adf'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:03.277078 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] The token cache is being invalidate because project ec20f75806154fa3a7d4bd0ea1bb6fee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:12:03.277521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:03.278226 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ef99f353-4202-4c72-9175-0a2ea5a58adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:03.280966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1378/2771] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:12:03 2022] DELETE /identity/v3/projects/ec20f75806154fa3a7d4bd0ea1bb6fee => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:12:03.287418 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc15d6-4d18-495b-8e36-888a51a8d887 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.288341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc15d6-4d18-495b-8e36-888a51a8d887 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.288341 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc15d6-4d18-495b-8e36-888a51a8d887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.317476 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-39dc15d6-4d18-495b-8e36-888a51a8d887 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:03.318925 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-39dc15d6-4d18-495b-8e36-888a51a8d887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:03.395419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1394/2772] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:03.404013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4e53-858f-494d-8027-6ad57d324ecb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.404411 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4e53-858f-494d-8027-6ad57d324ecb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.404766 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4e53-858f-494d-8027-6ad57d324ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.424755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b75b4e53-858f-494d-8027-6ad57d324ecb None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:03.426832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b75b4e53-858f-494d-8027-6ad57d324ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:03.434915 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1379/2773] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:12:03.439195 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc15d6-4d18-495b-8e36-888a51a8d887 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:03.451053 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc15d6-4d18-495b-8e36-888a51a8d887 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:03.455960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:03.456669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.456959 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.457610 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.458532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:12:03.458849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:03.460513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:03.474267 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1395/2774] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:12:03.478547 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b4e53-858f-494d-8027-6ad57d324ecb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:03.488593 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b4e53-858f-494d-8027-6ad57d324ecb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:03.494134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca696815-0dbd-41f5-b2ec-6fe752710c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:03.494832 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca696815-0dbd-41f5-b2ec-6fe752710c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.495119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca696815-0dbd-41f5-b2ec-6fe752710c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.495400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca696815-0dbd-41f5-b2ec-6fe752710c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.496109 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca696815-0dbd-41f5-b2ec-6fe752710c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:03.498068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca696815-0dbd-41f5-b2ec-6fe752710c76 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:03.533368 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1380/2775] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:12:03 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:12:03.538080 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:03.548557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcb89a-16b7-4618-a84e-c04fec9b80cc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:03.598800 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:03.611097 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4f5b4-bb75-4777-ab77-79d4c1e7a977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:03.611785 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4f5b4-bb75-4777-ab77-79d4c1e7a977 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.612299 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4f5b4-bb75-4777-ab77-79d4c1e7a977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.612596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4f5b4-bb75-4777-ab77-79d4c1e7a977 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.614286 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4f5b4-bb75-4777-ab77-79d4c1e7a977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:03.616246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4f5b4-bb75-4777-ab77-79d4c1e7a977 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:03.635565 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1396/2776] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:12:03 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:12:03.640791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca696815-0dbd-41f5-b2ec-6fe752710c76 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:03.652376 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca696815-0dbd-41f5-b2ec-6fe752710c76 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:03.657093 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdce49-b81e-427f-956f-df860b791ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:03.657758 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdce49-b81e-427f-956f-df860b791ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.658046 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdce49-b81e-427f-956f-df860b791ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.658329 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdce49-b81e-427f-956f-df860b791ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.659058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdce49-b81e-427f-956f-df860b791ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:03.661370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdce49-b81e-427f-956f-df860b791ff6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:03.672369 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1381/2777] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:12:03.676398 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4f5b4-bb75-4777-ab77-79d4c1e7a977 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:03.686777 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4f5b4-bb75-4777-ab77-79d4c1e7a977 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:03.691832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade7a31-0c2e-4045-9153-1f1c7c8c4cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:03.692556 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade7a31-0c2e-4045-9153-1f1c7c8c4cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.692846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade7a31-0c2e-4045-9153-1f1c7c8c4cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.693129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade7a31-0c2e-4045-9153-1f1c7c8c4cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.693867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade7a31-0c2e-4045-9153-1f1c7c8c4cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:03.695477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade7a31-0c2e-4045-9153-1f1c7c8c4cd1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:03.710446 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1397/2778] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:12:03.714680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdce49-b81e-427f-956f-df860b791ff6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:03.725266 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdce49-b81e-427f-956f-df860b791ff6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:03.729886 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:03.730767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.731052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.731331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] PATH_INFO: `/v3/projects/c2c1b13c34074936b0d56e0b04b95fc4/users/6368756ab9d946ddbd2e89ecf898e425/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.747091 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c1b13c34074936b0d56e0b04b95fc4, user_id=6368756ab9d946ddbd2e89ecf898e425, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:03.749762 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:03.752805 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:03.765160 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1382/2779] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:12:03 2022] PUT /identity/v3/projects/c2c1b13c34074936b0d56e0b04b95fc4/users/6368756ab9d946ddbd2e89ecf898e425/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:03.776287 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f9475-9f21-4abb-8fb2-3710c18d1d71 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.776587 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f9475-9f21-4abb-8fb2-3710c18d1d71 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.776897 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f9475-9f21-4abb-8fb2-3710c18d1d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.804844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b48f9475-9f21-4abb-8fb2-3710c18d1d71 None None] MFA Rules not processed for user `6368756ab9d946ddbd2e89ecf898e425`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:03.806100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b48f9475-9f21-4abb-8fb2-3710c18d1d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:03.876930 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1398/2780] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:03.888409 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:03.890141 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:03.946505 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775147c-c333-4710-af58-f08ed0d96c85 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:03.950173 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4cc70-00d9-497a-aebe-df8d2278225f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad37bdce39f64836b0cd14613cabe91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:03.950818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4cc70-00d9-497a-aebe-df8d2278225f service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:03.951103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4cc70-00d9-497a-aebe-df8d2278225f service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:03.951380 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4cc70-00d9-497a-aebe-df8d2278225f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:03.960303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4cc70-00d9-497a-aebe-df8d2278225f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:03.961723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4cc70-00d9-497a-aebe-df8d2278225f service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:03.970726 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1383/2781] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:04.011520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f9475-9f21-4abb-8fb2-3710c18d1d71 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:04.012935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b48f9475-9f21-4abb-8fb2-3710c18d1d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:04.084738 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f9475-9f21-4abb-8fb2-3710c18d1d71 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:04.089707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:04.090588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:04.090872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:04.091151 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] PATH_INFO: `/v3/users/4a3e23ac329048999100946fe57e6e6d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:04.092534 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a3e23ac329048999100946fe57e6e6d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:04.094358 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:04.145736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a3e23ac329048999100946fe57e6e6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.157501 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:04.160114 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a3e23ac329048999100946fe57e6e6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.161090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a3e23ac329048999100946fe57e6e6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.161971 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a3e23ac329048999100946fe57e6e6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.171037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a3e23ac329048999100946fe57e6e6d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.184519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1399/2782] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:12:04 2022] DELETE /identity/v3/users/4a3e23ac329048999100946fe57e6e6d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:04.189383 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863fba8-b775-40bf-8b39-610e6d98a374 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:04.189512 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863fba8-b775-40bf-8b39-610e6d98a374 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:04.189740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863fba8-b775-40bf-8b39-610e6d98a374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:04.208180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8863fba8-b775-40bf-8b39-610e6d98a374 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:04.209395 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8863fba8-b775-40bf-8b39-610e6d98a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:04.275867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1384/2783] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:04.286866 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:04.288551 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:04.355872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d67002-f541-4ef9-ab6b-6909b0625c2b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:04.359552 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:04.360232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:04.360523 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:04.360957 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:04.373852 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:04.375430 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:04.388400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1400/2784] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:04.709537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863fba8-b775-40bf-8b39-610e6d98a374 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:04.711014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8863fba8-b775-40bf-8b39-610e6d98a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:04.727441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863fba8-b775-40bf-8b39-610e6d98a374 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:04.732603 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:04.733626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:04.734082 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:04.734548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] PATH_INFO: `/v3/projects/0d5f8209ebbd42a4897c2a77a3e80cda` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:04.736063 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5f8209ebbd42a4897c2a77a3e80cda)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:04.739074 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:04.797447 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5f8209ebbd42a4897c2a77a3e80cda', 'request_id': 'req-f8e2ba2a-427f-41be-a7bd-dc820854303e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.805997 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5f8209ebbd42a4897c2a77a3e80cda', 'request_id': 'req-f8e2ba2a-427f-41be-a7bd-dc820854303e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.811852 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5f8209ebbd42a4897c2a77a3e80cda', 'request_id': 'req-f8e2ba2a-427f-41be-a7bd-dc820854303e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.826370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] The token cache is being invalidate because project 0d5f8209ebbd42a4897c2a77a3e80cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:12:04.826588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.827227 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f8e2ba2a-427f-41be-a7bd-dc820854303e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:04.828849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1385/2785] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:12:04 2022] DELETE /identity/v3/projects/0d5f8209ebbd42a4897c2a77a3e80cda => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:05.673805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:05.675236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:05.688911 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b581fb-a192-4cae-bb6e-387e972f60e7 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:05.750830 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:05.751708 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:05.751999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:05.752321 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] PATH_INFO: `/v3/users/94fb4526e77e4c00bf9da389c04d7da2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:05.753521 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94fb4526e77e4c00bf9da389c04d7da2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:05.755467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:05.804001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94fb4526e77e4c00bf9da389c04d7da2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:05.815918 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94fb4526e77e4c00bf9da389c04d7da2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:05.816727 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94fb4526e77e4c00bf9da389c04d7da2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:05.817467 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94fb4526e77e4c00bf9da389c04d7da2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:05.825604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94fb4526e77e4c00bf9da389c04d7da2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:05.836185 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:05.840367 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1401/2786] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:12:05 2022] DELETE /identity/v3/users/94fb4526e77e4c00bf9da389c04d7da2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:05.845076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38795b4c-873d-46e6-96fd-f18dd405b75f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:05.845597 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38795b4c-873d-46e6-96fd-f18dd405b75f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:05.845743 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38795b4c-873d-46e6-96fd-f18dd405b75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:05.865231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-38795b4c-873d-46e6-96fd-f18dd405b75f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:05.866476 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-38795b4c-873d-46e6-96fd-f18dd405b75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:05.883889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:05.929513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1386/2787] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:05.939724 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:05.941157 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:06.001238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd286b-1ba5-4995-8ec4-acb178832f12 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:06.005245 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab890ea3-300c-404d-b38f-100e39c2279d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:06.005892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab890ea3-300c-404d-b38f-100e39c2279d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.006178 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab890ea3-300c-404d-b38f-100e39c2279d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.006456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab890ea3-300c-404d-b38f-100e39c2279d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.015482 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab890ea3-300c-404d-b38f-100e39c2279d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:06.016998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab890ea3-300c-404d-b38f-100e39c2279d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:06.028596 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1402/2788] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:06.321631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38795b4c-873d-46e6-96fd-f18dd405b75f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:06.323080 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-38795b4c-873d-46e6-96fd-f18dd405b75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:06.335995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38795b4c-873d-46e6-96fd-f18dd405b75f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:06.340768 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:06.341719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.342023 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.342283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] PATH_INFO: `/v3/projects/9bee19f42aa14e998c676a4b951dd58c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.343495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bee19f42aa14e998c676a4b951dd58c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:06.345467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:06.400052 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bee19f42aa14e998c676a4b951dd58c', 'request_id': 'req-3d12fa1f-1846-467d-9340-7f84e35d9ac3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:06.406324 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bee19f42aa14e998c676a4b951dd58c', 'request_id': 'req-3d12fa1f-1846-467d-9340-7f84e35d9ac3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:06.412139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bee19f42aa14e998c676a4b951dd58c', 'request_id': 'req-3d12fa1f-1846-467d-9340-7f84e35d9ac3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:06.424748 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] The token cache is being invalidate because project 9bee19f42aa14e998c676a4b951dd58c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:12:06.425038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:06.425673 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3d12fa1f-1846-467d-9340-7f84e35d9ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:06.427495 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1387/2789] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:12:06 2022] DELETE /identity/v3/projects/9bee19f42aa14e998c676a4b951dd58c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:06.432496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675780b-4d16-46d8-bf72-d7bc045138f3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.433257 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675780b-4d16-46d8-bf72-d7bc045138f3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.433375 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675780b-4d16-46d8-bf72-d7bc045138f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.454021 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c675780b-4d16-46d8-bf72-d7bc045138f3 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:06.455493 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c675780b-4d16-46d8-bf72-d7bc045138f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:06.533130 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1403/2790] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:06.541182 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c4d43-4f53-41ea-ac62-433b2788c20d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.541486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c4d43-4f53-41ea-ac62-433b2788c20d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.541799 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c4d43-4f53-41ea-ac62-433b2788c20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.561448 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d94c4d43-4f53-41ea-ac62-433b2788c20d None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:06.562708 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d94c4d43-4f53-41ea-ac62-433b2788c20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:06.571943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1388/2791] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:06.576404 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675780b-4d16-46d8-bf72-d7bc045138f3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:06.587005 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675780b-4d16-46d8-bf72-d7bc045138f3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:06.592133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:06.592823 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.595411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.595822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.597434 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:12:06.598260 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:06.601241 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:06.617522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1404/2792] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:12:06.622496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c4d43-4f53-41ea-ac62-433b2788c20d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:06.632252 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c4d43-4f53-41ea-ac62-433b2788c20d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:06.637248 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92602db-a51c-46c0-91ee-51c210dd3edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:06.637904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92602db-a51c-46c0-91ee-51c210dd3edc admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.638190 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92602db-a51c-46c0-91ee-51c210dd3edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.638467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92602db-a51c-46c0-91ee-51c210dd3edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.639148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92602db-a51c-46c0-91ee-51c210dd3edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:06.641387 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92602db-a51c-46c0-91ee-51c210dd3edc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:06.666691 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1389/2793] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:12:06 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:12:06.670806 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:06.681483 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989e70-af78-43d6-b62d-59526795e5d8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:06.746471 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e32a2-1493-4ccd-8305-3d08fb9974f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:06.747062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e32a2-1493-4ccd-8305-3d08fb9974f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.747368 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e32a2-1493-4ccd-8305-3d08fb9974f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.747629 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e32a2-1493-4ccd-8305-3d08fb9974f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.748360 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e32a2-1493-4ccd-8305-3d08fb9974f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:06.750293 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e32a2-1493-4ccd-8305-3d08fb9974f1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:06.771956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1405/2794] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:12:06 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:12:06.776204 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92602db-a51c-46c0-91ee-51c210dd3edc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:06.786838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92602db-a51c-46c0-91ee-51c210dd3edc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:06.791738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab48f4c-418b-429b-ab3a-347e45498461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:06.792501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab48f4c-418b-429b-ab3a-347e45498461 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.792784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab48f4c-418b-429b-ab3a-347e45498461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.793087 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab48f4c-418b-429b-ab3a-347e45498461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.793842 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab48f4c-418b-429b-ab3a-347e45498461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:06.795453 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab48f4c-418b-429b-ab3a-347e45498461 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:06.809263 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1390/2795] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:12:06.813450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e32a2-1493-4ccd-8305-3d08fb9974f1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:06.824473 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e32a2-1493-4ccd-8305-3d08fb9974f1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:06.829238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb9776-e057-4e48-aa8d-a2a0d389a4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:06.829899 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb9776-e057-4e48-aa8d-a2a0d389a4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.830188 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb9776-e057-4e48-aa8d-a2a0d389a4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.830462 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb9776-e057-4e48-aa8d-a2a0d389a4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.831183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb9776-e057-4e48-aa8d-a2a0d389a4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:06.833101 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb9776-e057-4e48-aa8d-a2a0d389a4b0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:06.843072 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1406/2796] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:12:06.847171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab48f4c-418b-429b-ab3a-347e45498461 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:06.857626 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab48f4c-418b-429b-ab3a-347e45498461 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:06.862423 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:06.863995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.868244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.868244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] PATH_INFO: `/v3/projects/9020bb4044f74e2ab239a70f0fda9dc4/users/e447e7dc4aa24a5e936d0986c127dfd3/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.887637 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9020bb4044f74e2ab239a70f0fda9dc4, user_id=e447e7dc4aa24a5e936d0986c127dfd3, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:06.890487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:06.903814 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1391/2797] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:12:06 2022] PUT /identity/v3/projects/9020bb4044f74e2ab239a70f0fda9dc4/users/e447e7dc4aa24a5e936d0986c127dfd3/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:06.915126 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea572ff-4f92-4246-afd1-18de123a2411 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:06.915419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea572ff-4f92-4246-afd1-18de123a2411 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:06.915705 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea572ff-4f92-4246-afd1-18de123a2411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:06.938405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:06.949395 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fea572ff-4f92-4246-afd1-18de123a2411 None None] MFA Rules not processed for user `e447e7dc4aa24a5e936d0986c127dfd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:06.950934 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fea572ff-4f92-4246-afd1-18de123a2411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:07.015436 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1407/2798] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:07.028860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:07.030951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:07.078522 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:07.091677 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8a61e-5f86-4ccc-b3ed-3cb289abdcd0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:07.095360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c4b44-9530-4322-be05-80d80d378769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:07.096012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c4b44-9530-4322-be05-80d80d378769 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:07.096331 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c4b44-9530-4322-be05-80d80d378769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:07.096611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c4b44-9530-4322-be05-80d80d378769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:07.105431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c4b44-9530-4322-be05-80d80d378769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:07.106863 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c4b44-9530-4322-be05-80d80d378769 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:07.116406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1392/2799] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:09.701728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea572ff-4f92-4246-afd1-18de123a2411 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:09.764799 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea572ff-4f92-4246-afd1-18de123a2411 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:09.770345 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:09.771333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:09.771633 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:09.771917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] PATH_INFO: `/v3/users/e447e7dc4aa24a5e936d0986c127dfd3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:09.773167 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e447e7dc4aa24a5e936d0986c127dfd3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:09.775165 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:09.828465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e447e7dc4aa24a5e936d0986c127dfd3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:09.843605 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e447e7dc4aa24a5e936d0986c127dfd3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:09.844730 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e447e7dc4aa24a5e936d0986c127dfd3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:09.845301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e447e7dc4aa24a5e936d0986c127dfd3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:09.853196 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e447e7dc4aa24a5e936d0986c127dfd3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:09.866026 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1408/2800] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:12:09 2022] DELETE /identity/v3/users/e447e7dc4aa24a5e936d0986c127dfd3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:09.870845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640bff1-5227-4f9c-b307-97f97a88c52f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:09.871357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640bff1-5227-4f9c-b307-97f97a88c52f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:09.871456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640bff1-5227-4f9c-b307-97f97a88c52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:09.889467 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0640bff1-5227-4f9c-b307-97f97a88c52f None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:09.890698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0640bff1-5227-4f9c-b307-97f97a88c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:09.959228 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1393/2801] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:12:09.969166 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:09.970584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:09.992529 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:10.034558 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eefadd-c148-4e89-88fd-954d30bfcb70 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:10.038337 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f295f-5194-40f2-8282-ca7f844348b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:10.038984 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f295f-5194-40f2-8282-ca7f844348b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.039272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f295f-5194-40f2-8282-ca7f844348b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.039583 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f295f-5194-40f2-8282-ca7f844348b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.048676 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f295f-5194-40f2-8282-ca7f844348b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:10.050109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f295f-5194-40f2-8282-ca7f844348b2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:10.061542 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1409/2802] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:10.348088 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640bff1-5227-4f9c-b307-97f97a88c52f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:10.349934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0640bff1-5227-4f9c-b307-97f97a88c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:10.362391 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640bff1-5227-4f9c-b307-97f97a88c52f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:10.370244 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:10.371755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.372311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.372825 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] PATH_INFO: `/v3/projects/9020bb4044f74e2ab239a70f0fda9dc4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.374942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9020bb4044f74e2ab239a70f0fda9dc4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:10.378784 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:10.443330 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9020bb4044f74e2ab239a70f0fda9dc4', 'request_id': 'req-5cf57d72-bbbf-4ffa-86db-ee280e66733b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:10.449695 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9020bb4044f74e2ab239a70f0fda9dc4', 'request_id': 'req-5cf57d72-bbbf-4ffa-86db-ee280e66733b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:10.455309 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9020bb4044f74e2ab239a70f0fda9dc4', 'request_id': 'req-5cf57d72-bbbf-4ffa-86db-ee280e66733b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:10.467858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] The token cache is being invalidate because project 9020bb4044f74e2ab239a70f0fda9dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:12:10.468174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:10.468815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5cf57d72-bbbf-4ffa-86db-ee280e66733b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:10.470366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1394/2803] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:12:10 2022] DELETE /identity/v3/projects/9020bb4044f74e2ab239a70f0fda9dc4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:10.475715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82f616-bd94-4202-af80-0cd4b90802e0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.476012 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82f616-bd94-4202-af80-0cd4b90802e0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.476335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82f616-bd94-4202-af80-0cd4b90802e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.494851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cc82f616-bd94-4202-af80-0cd4b90802e0 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:10.496062 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cc82f616-bd94-4202-af80-0cd4b90802e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:10.562929 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1410/2804] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:10.571738 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c1e0-b599-4841-bbc4-1212ee3b3486 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.572030 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c1e0-b599-4841-bbc4-1212ee3b3486 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.572339 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6c1e0-b599-4841-bbc4-1212ee3b3486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.591059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67a6c1e0-b599-4841-bbc4-1212ee3b3486 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:10.592290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67a6c1e0-b599-4841-bbc4-1212ee3b3486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:10.600782 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1395/2805] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:10.604983 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82f616-bd94-4202-af80-0cd4b90802e0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:10.615524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82f616-bd94-4202-af80-0cd4b90802e0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:10.620183 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:10.620846 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.621135 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.621419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.622176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:12:10.622490 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:10.624117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:10.636859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1411/2806] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:12:10.643543 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6c1e0-b599-4841-bbc4-1212ee3b3486 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:10.654668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6c1e0-b599-4841-bbc4-1212ee3b3486 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:10.659731 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9287-1473-41b4-931d-f7445a9cfdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:10.660578 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9287-1473-41b4-931d-f7445a9cfdc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.661089 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9287-1473-41b4-931d-f7445a9cfdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.661441 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9287-1473-41b4-931d-f7445a9cfdc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.662148 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e9287-1473-41b4-931d-f7445a9cfdc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:10.664068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e9287-1473-41b4-931d-f7445a9cfdc6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:10.691425 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1396/2807] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:12:10 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:12:10.696297 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:10.706371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bc4da-ffda-4242-b619-bfa46bc3e6de admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:10.773822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7450c03-528c-4d4b-870a-955a01365ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:10.774507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7450c03-528c-4d4b-870a-955a01365ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.774793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7450c03-528c-4d4b-870a-955a01365ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.775075 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7450c03-528c-4d4b-870a-955a01365ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.775768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7450c03-528c-4d4b-870a-955a01365ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:10.777803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7450c03-528c-4d4b-870a-955a01365ac5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:10.798729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1412/2808] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:12:10 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:12:10.803469 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9287-1473-41b4-931d-f7445a9cfdc6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:10.813874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9287-1473-41b4-931d-f7445a9cfdc6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:10.818853 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bfb76-fdab-483f-b573-c725ba573989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:10.819513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bfb76-fdab-483f-b573-c725ba573989 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.819798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bfb76-fdab-483f-b573-c725ba573989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.820076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bfb76-fdab-483f-b573-c725ba573989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.820845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bfb76-fdab-483f-b573-c725ba573989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:10.822422 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bfb76-fdab-483f-b573-c725ba573989 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:10.832874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1397/2809] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:12:10.836927 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7450c03-528c-4d4b-870a-955a01365ac5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:10.846884 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7450c03-528c-4d4b-870a-955a01365ac5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:10.852229 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619361f5-eaab-44ef-b732-e80e4de83055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:10.852906 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619361f5-eaab-44ef-b732-e80e4de83055 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.853199 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619361f5-eaab-44ef-b732-e80e4de83055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.853484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619361f5-eaab-44ef-b732-e80e4de83055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.854219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619361f5-eaab-44ef-b732-e80e4de83055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:10.855836 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619361f5-eaab-44ef-b732-e80e4de83055 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:10.867494 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1413/2810] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:12:10.871767 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bfb76-fdab-483f-b573-c725ba573989 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:10.888194 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bfb76-fdab-483f-b573-c725ba573989 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:10.894700 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:10.895632 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.895916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.896249 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] PATH_INFO: `/v3/projects/61a881aaa8884bac81500ab8d432d560/users/fb5d88eb651d4c00b6ff1f116b3180ef/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.913532 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a881aaa8884bac81500ab8d432d560, user_id=fb5d88eb651d4c00b6ff1f116b3180ef, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:10.916216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:10.931098 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1398/2811] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:12:10 2022] PUT /identity/v3/projects/61a881aaa8884bac81500ab8d432d560/users/fb5d88eb651d4c00b6ff1f116b3180ef/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:10.941338 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547884a-193e-42e1-828e-d1cf17f05f7e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:10.941651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547884a-193e-42e1-828e-d1cf17f05f7e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:10.941999 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547884a-193e-42e1-828e-d1cf17f05f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:10.971120 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4547884a-193e-42e1-828e-d1cf17f05f7e None None] MFA Rules not processed for user `fb5d88eb651d4c00b6ff1f116b3180ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:10.972419 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4547884a-193e-42e1-828e-d1cf17f05f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:10.985967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:11.050212 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1414/2812] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 30 12:12:11.062318 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:11.064010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:11.070658 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:11.122917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19bd55-5af2-4b76-97c1-ab5e7895fa9e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:11.126687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcbdbe-7b4f-4c05-8e91-4a4e9384b63b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c3c27e908e34977b248042f96c70dd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:11.127337 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcbdbe-7b4f-4c05-8e91-4a4e9384b63b service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:11.127621 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcbdbe-7b4f-4c05-8e91-4a4e9384b63b service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:11.127901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcbdbe-7b4f-4c05-8e91-4a4e9384b63b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:11.136620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdcbdbe-7b4f-4c05-8e91-4a4e9384b63b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:11.138048 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdcbdbe-7b4f-4c05-8e91-4a4e9384b63b service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:11.147928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1399/2813] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:11.180914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547884a-193e-42e1-828e-d1cf17f05f7e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:11.244068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547884a-193e-42e1-828e-d1cf17f05f7e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:11.248665 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:11.249536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:11.249817 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:11.250090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] PATH_INFO: `/v3/users/fb5d88eb651d4c00b6ff1f116b3180ef` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:11.251289 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb5d88eb651d4c00b6ff1f116b3180ef)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:11.253263 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:11.303156 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb5d88eb651d4c00b6ff1f116b3180ef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.317234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb5d88eb651d4c00b6ff1f116b3180ef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.317982 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb5d88eb651d4c00b6ff1f116b3180ef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.318600 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb5d88eb651d4c00b6ff1f116b3180ef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.325724 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb5d88eb651d4c00b6ff1f116b3180ef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.339272 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1415/2814] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:12:11 2022] DELETE /identity/v3/users/fb5d88eb651d4c00b6ff1f116b3180ef => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:11.345150 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd7495-e336-4324-8f2a-f3128129ceec None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:11.345530 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd7495-e336-4324-8f2a-f3128129ceec None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:11.345858 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd7495-e336-4324-8f2a-f3128129ceec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:11.364798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-90dd7495-e336-4324-8f2a-f3128129ceec None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:11.366041 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-90dd7495-e336-4324-8f2a-f3128129ceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:11.428431 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1400/2815] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:11.438438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:11.440052 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:11.481512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:11.500687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a83af7-09b2-4611-93c1-f9537affb48c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:11.505073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:11.505733 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:11.506029 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:11.506306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:11.515684 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:11.517182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:11.527955 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1416/2816] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:11.813095 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd7495-e336-4324-8f2a-f3128129ceec None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:11.814633 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-90dd7495-e336-4324-8f2a-f3128129ceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:11.830599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd7495-e336-4324-8f2a-f3128129ceec None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:11.839643 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:11.840596 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:11.840923 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:11.841207 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] PATH_INFO: `/v3/projects/61a881aaa8884bac81500ab8d432d560` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:11.842464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a881aaa8884bac81500ab8d432d560)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:11.845340 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:11.904831 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a881aaa8884bac81500ab8d432d560', 'request_id': 'req-f3e14f09-7b86-4790-97a9-fded575b1e02'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.911999 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a881aaa8884bac81500ab8d432d560', 'request_id': 'req-f3e14f09-7b86-4790-97a9-fded575b1e02'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.917721 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a881aaa8884bac81500ab8d432d560', 'request_id': 'req-f3e14f09-7b86-4790-97a9-fded575b1e02'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.930615 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] The token cache is being invalidate because project 61a881aaa8884bac81500ab8d432d560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:12:11.930838 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.931463 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f3e14f09-7b86-4790-97a9-fded575b1e02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:11.933047 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1401/2817] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:12:11 2022] DELETE /identity/v3/projects/61a881aaa8884bac81500ab8d432d560 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:12:21.523513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:21.927407 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:30.744996 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:30.746444 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:30.759650 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f881302-1f62-46ce-a806-c905eec41e01 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:30.817638 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:30.818510 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:30.818796 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:30.819091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] PATH_INFO: `/v3/users/6368756ab9d946ddbd2e89ecf898e425` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:30.820344 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6368756ab9d946ddbd2e89ecf898e425)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:30.822282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:30.874945 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6368756ab9d946ddbd2e89ecf898e425'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:30.887278 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6368756ab9d946ddbd2e89ecf898e425'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:30.887977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6368756ab9d946ddbd2e89ecf898e425'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:30.888644 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6368756ab9d946ddbd2e89ecf898e425'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:30.895835 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6368756ab9d946ddbd2e89ecf898e425'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:30.909145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1417/2818] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:12:30 2022] DELETE /identity/v3/users/6368756ab9d946ddbd2e89ecf898e425 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:30.914264 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2030e7-f661-4375-8e06-a9cd8aa3a0b6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:30.914787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2030e7-f661-4375-8e06-a9cd8aa3a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:30.914888 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2030e7-f661-4375-8e06-a9cd8aa3a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:30.932650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6f2030e7-f661-4375-8e06-a9cd8aa3a0b6 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:12:30.933870 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6f2030e7-f661-4375-8e06-a9cd8aa3a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:30.999650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1402/2819] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:12:31.009393 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:31.011666 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:31.069526 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17de03-7676-4a43-a769-cb432a5991fb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:31.073282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:31.073940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:31.074588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:31.074588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:31.084576 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:31.086520 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:31.096422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1418/2820] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:12:31.496492 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2030e7-f661-4375-8e06-a9cd8aa3a0b6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:12:31.497995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6f2030e7-f661-4375-8e06-a9cd8aa3a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:12:31.511169 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2030e7-f661-4375-8e06-a9cd8aa3a0b6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:12:31.516302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:12:31.517200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:12:31.517735 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:12:31.518010 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] PATH_INFO: `/v3/projects/c2c1b13c34074936b0d56e0b04b95fc4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:12:31.519213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2c1b13c34074936b0d56e0b04b95fc4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:12:31.521231 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:12:31.568964 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c1b13c34074936b0d56e0b04b95fc4', 'request_id': 'req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:31.575791 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c1b13c34074936b0d56e0b04b95fc4', 'request_id': 'req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:31.581592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c1b13c34074936b0d56e0b04b95fc4', 'request_id': 'req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:31.595004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] The token cache is being invalidate because project c2c1b13c34074936b0d56e0b04b95fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:12:31.595294 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:31.595934 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-07204cb1-b9b8-4b9c-8b1d-b2daa197830d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:12:31.597542 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1403/2821] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:12:31 2022] DELETE /identity/v3/projects/c2c1b13c34074936b0d56e0b04b95fc4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:12:41.091943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:12:41.591830 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:13:12.967424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:12.968889 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:13.041646 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40b68-1452-426a-8f94-2d1176eb20fd service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:13.106414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:13.107100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:13.107390 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:13.107671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:13.109497 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:13.121099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:13.122710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:13.132763 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1419/2822] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:13:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:13:23.128465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=58 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:13:26.047242 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255f0a4-4814-4e36-a88c-1caa0d7aa8f5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:26.047769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255f0a4-4814-4e36-a88c-1caa0d7aa8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:26.047860 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255f0a4-4814-4e36-a88c-1caa0d7aa8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:26.068232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9255f0a4-4814-4e36-a88c-1caa0d7aa8f5 None None] MFA Rules not processed for user `d087549c76ae4c3896c7c8c6b5a74363`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:13:26.069515 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9255f0a4-4814-4e36-a88c-1caa0d7aa8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:26.128602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1404/2823] 10.222.0.32 () {58 vars in 967 bytes} [Mon May 30 12:13:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:13:26.179038 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:26.180472 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:26.249778 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fdc3c-f21e-4fbd-b7a1-501a6450c505 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:26.254791 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed79aa-29e9-46ca-b120-2ba4a8f67255 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0cb45c1eb6040548634022673a51e56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:26.255440 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed79aa-29e9-46ca-b120-2ba4a8f67255 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:26.255729 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed79aa-29e9-46ca-b120-2ba4a8f67255 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:26.256011 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed79aa-29e9-46ca-b120-2ba4a8f67255 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:26.264385 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed79aa-29e9-46ca-b120-2ba4a8f67255 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:26.265998 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed79aa-29e9-46ca-b120-2ba4a8f67255 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:26.274397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1420/2824] 10.222.0.32 () {60 vars in 1384 bytes} [Mon May 30 12:13:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 30 12:13:33.014874 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255f0a4-4814-4e36-a88c-1caa0d7aa8f5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:33.016963 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9255f0a4-4814-4e36-a88c-1caa0d7aa8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:33.086761 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255f0a4-4814-4e36-a88c-1caa0d7aa8f5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:33.092005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:33.092933 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:33.093232 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:33.093501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] PATH_INFO: `/v3/users/c8915c74c4d24d2cb1cded5aa7927120` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:33.094720 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8915c74c4d24d2cb1cded5aa7927120)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:33.096687 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:33.142817 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8915c74c4d24d2cb1cded5aa7927120'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.154972 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8915c74c4d24d2cb1cded5aa7927120'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.155668 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8915c74c4d24d2cb1cded5aa7927120'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.162486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8915c74c4d24d2cb1cded5aa7927120'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.163111 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8915c74c4d24d2cb1cded5aa7927120'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.177460 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1405/2825] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:13:33 2022] DELETE /identity/v3/users/c8915c74c4d24d2cb1cded5aa7927120 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:13:33.363969 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed79aa-29e9-46ca-b120-2ba4a8f67255 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:33.376833 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed79aa-29e9-46ca-b120-2ba4a8f67255 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:33.442438 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:33.443308 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:33.443598 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:33.443880 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] PATH_INFO: `/v3/projects/dc19bd60cde244559a823a47293540f7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:33.445104 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc19bd60cde244559a823a47293540f7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:33.447636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:33.495670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc19bd60cde244559a823a47293540f7', 'request_id': 'req-d1d03546-9dc9-475a-b037-b51ae6d5385e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.503247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc19bd60cde244559a823a47293540f7', 'request_id': 'req-d1d03546-9dc9-475a-b037-b51ae6d5385e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.510511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc19bd60cde244559a823a47293540f7', 'request_id': 'req-d1d03546-9dc9-475a-b037-b51ae6d5385e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.522279 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] The token cache is being invalidate because project dc19bd60cde244559a823a47293540f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:13:33.522509 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.523192 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.525400 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1421/2826] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:13:33 2022] DELETE /identity/v3/projects/dc19bd60cde244559a823a47293540f7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:13:33.529629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:33.531053 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:33.544537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde55c96-49e8-49f8-83a8-fc6556d8261b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:33.564711 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:13:33.607620 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:33.608602 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:33.608930 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:33.609253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] PATH_INFO: `/v3/users/02202bbdf9014e0284db9f6ab61d1480` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:33.610623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02202bbdf9014e0284db9f6ab61d1480)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:33.612760 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:33.660332 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02202bbdf9014e0284db9f6ab61d1480'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.672534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02202bbdf9014e0284db9f6ab61d1480'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.673222 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02202bbdf9014e0284db9f6ab61d1480'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.680586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02202bbdf9014e0284db9f6ab61d1480'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.681349 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b42bb092-2c54-4961-adbb-5967be5979d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02202bbdf9014e0284db9f6ab61d1480'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:33.694119 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1406/2827] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:13:33 2022] DELETE /identity/v3/users/02202bbdf9014e0284db9f6ab61d1480 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:13:33.877176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:33.879567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:33.914771 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03546-9dc9-475a-b037-b51ae6d5385e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:33.933182 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:13:33.977149 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:33.978158 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:33.978452 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:33.978736 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] PATH_INFO: `/v3/projects/20c153c20fd24882972be249a215ac2c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:33.979940 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c153c20fd24882972be249a215ac2c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:33.981901 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:34.031775 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c153c20fd24882972be249a215ac2c', 'request_id': 'req-774cb68f-46a5-4a22-ab37-a689dde3d699'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:34.039793 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c153c20fd24882972be249a215ac2c', 'request_id': 'req-774cb68f-46a5-4a22-ab37-a689dde3d699'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:34.046660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c153c20fd24882972be249a215ac2c', 'request_id': 'req-774cb68f-46a5-4a22-ab37-a689dde3d699'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:34.057808 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] The token cache is being invalidate because project 20c153c20fd24882972be249a215ac2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:13:34.058100 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:34.058768 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-774cb68f-46a5-4a22-ab37-a689dde3d699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:13:34.060450 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1422/2828] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:13:33 2022] DELETE /identity/v3/projects/20c153c20fd24882972be249a215ac2c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:13:34.065451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15162f-38ea-4c0b-bedc-823524c72b34 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.065942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15162f-38ea-4c0b-bedc-823524c72b34 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.066042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15162f-38ea-4c0b-bedc-823524c72b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.084937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0e15162f-38ea-4c0b-bedc-823524c72b34 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:13:34.086145 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0e15162f-38ea-4c0b-bedc-823524c72b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:34.146636 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1407/2829] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:13:34.154787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c686af0-58e6-4258-9f44-4eaa28952a74 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.155081 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c686af0-58e6-4258-9f44-4eaa28952a74 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.155363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c686af0-58e6-4258-9f44-4eaa28952a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.173822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8c686af0-58e6-4258-9f44-4eaa28952a74 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:13:34.175014 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8c686af0-58e6-4258-9f44-4eaa28952a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:34.183642 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1423/2830] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:13:34.187801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15162f-38ea-4c0b-bedc-823524c72b34 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:34.198415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15162f-38ea-4c0b-bedc-823524c72b34 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:34.203042 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:34.203698 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.203985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.204304 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.205032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:13:34.205348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:34.206942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:34.218778 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1408/2831] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:13:34.223056 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c686af0-58e6-4258-9f44-4eaa28952a74 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:34.232863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c686af0-58e6-4258-9f44-4eaa28952a74 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:34.237591 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562856ea-3c1f-4245-ac28-a3e72ea89bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:34.238246 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562856ea-3c1f-4245-ac28-a3e72ea89bff admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.238532 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562856ea-3c1f-4245-ac28-a3e72ea89bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.238814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562856ea-3c1f-4245-ac28-a3e72ea89bff admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.239498 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562856ea-3c1f-4245-ac28-a3e72ea89bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:34.241432 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562856ea-3c1f-4245-ac28-a3e72ea89bff admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:34.267549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1424/2832] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:13:34 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:13:34.271904 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:34.281939 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53028d91-c4ac-4633-9458-51d57b982f4c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:34.311848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:13:34.344275 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617f89e-54a2-4fe0-b8b7-519c20611de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:34.344951 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f89e-54a2-4fe0-b8b7-519c20611de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.345239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f89e-54a2-4fe0-b8b7-519c20611de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.345514 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f89e-54a2-4fe0-b8b7-519c20611de0 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.346216 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617f89e-54a2-4fe0-b8b7-519c20611de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:34.348180 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617f89e-54a2-4fe0-b8b7-519c20611de0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:34.370924 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1409/2833] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:13:34 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:13:34.374985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562856ea-3c1f-4245-ac28-a3e72ea89bff admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:34.385645 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562856ea-3c1f-4245-ac28-a3e72ea89bff admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:34.390422 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb225c-4975-48fc-b23d-ffef02a63c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:34.391085 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb225c-4975-48fc-b23d-ffef02a63c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.391414 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb225c-4975-48fc-b23d-ffef02a63c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.391739 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb225c-4975-48fc-b23d-ffef02a63c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.392589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb225c-4975-48fc-b23d-ffef02a63c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:34.394648 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb225c-4975-48fc-b23d-ffef02a63c3b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:34.405857 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1425/2834] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 30 12:13:34.409794 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617f89e-54a2-4fe0-b8b7-519c20611de0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:34.420513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617f89e-54a2-4fe0-b8b7-519c20611de0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:34.425153 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8fb39-0d3a-4ed5-989d-4bb76fe003c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:34.425810 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8fb39-0d3a-4ed5-989d-4bb76fe003c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.426096 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8fb39-0d3a-4ed5-989d-4bb76fe003c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.426403 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8fb39-0d3a-4ed5-989d-4bb76fe003c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.427129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8fb39-0d3a-4ed5-989d-4bb76fe003c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:34.428747 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8fb39-0d3a-4ed5-989d-4bb76fe003c6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:34.441703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1410/2835] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:13:34.445812 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb225c-4975-48fc-b23d-ffef02a63c3b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:34.455890 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb225c-4975-48fc-b23d-ffef02a63c3b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:34.460524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:34.461408 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.461691 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.461967 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] PATH_INFO: `/v3/projects/7ab20a59f54348e6b0d892ea010aff9c/users/7fce4b25aa88412590502ccbc8ce0690/roles/932e429fa28248f285663a90fc880876` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.476425 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab20a59f54348e6b0d892ea010aff9c, user_id=7fce4b25aa88412590502ccbc8ce0690, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:34.479652 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:34.492281 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1426/2836] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:13:34 2022] PUT /identity/v3/projects/7ab20a59f54348e6b0d892ea010aff9c/users/7fce4b25aa88412590502ccbc8ce0690/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:13:34.497901 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841f6a-01a9-4f09-8b3e-4956c57617f4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.498221 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841f6a-01a9-4f09-8b3e-4956c57617f4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.498502 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841f6a-01a9-4f09-8b3e-4956c57617f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.523076 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-64841f6a-01a9-4f09-8b3e-4956c57617f4 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:13:34.524336 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-64841f6a-01a9-4f09-8b3e-4956c57617f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:34.585703 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1411/2837] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:13:34.595920 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:34.597682 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:34.654765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:13:34.660831 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037b278-5db1-4718-b78f-e7765f2aad6c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:34.665513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdc83b-e09b-4aac-9b58-692c218410ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:34.667152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdc83b-e09b-4aac-9b58-692c218410ce service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:34.667445 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdc83b-e09b-4aac-9b58-692c218410ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:34.667726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdc83b-e09b-4aac-9b58-692c218410ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:34.676895 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdc83b-e09b-4aac-9b58-692c218410ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:34.678353 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdc83b-e09b-4aac-9b58-692c218410ce service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:34.689228 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1427/2838] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:13:38.929826 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07341-8459-4606-9104-29bf564a113c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:38.930357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07341-8459-4606-9104-29bf564a113c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:38.930357 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07341-8459-4606-9104-29bf564a113c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:38.957948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-06f07341-8459-4606-9104-29bf564a113c None None] MFA Rules not processed for user `7fce4b25aa88412590502ccbc8ce0690`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:13:38.959181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-06f07341-8459-4606-9104-29bf564a113c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:39.042815 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1412/2839] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:13:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:13:39.047611 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdc83b-e09b-4aac-9b58-692c218410ce service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:39.059258 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdc83b-e09b-4aac-9b58-692c218410ce service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:39.064675 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82195f6f-4d4c-44dc-987a-d6a03c84af33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:39.065463 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195f6f-4d4c-44dc-987a-d6a03c84af33 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:39.065822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195f6f-4d4c-44dc-987a-d6a03c84af33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:39.066176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195f6f-4d4c-44dc-987a-d6a03c84af33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:39.066949 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82195f6f-4d4c-44dc-987a-d6a03c84af33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:39.069007 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82195f6f-4d4c-44dc-987a-d6a03c84af33 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:39.101039 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1428/2840] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:13:39 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:13:39.105779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07341-8459-4606-9104-29bf564a113c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:39.117580 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07341-8459-4606-9104-29bf564a113c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:39.161432 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:13:39.182873 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984400af-56e9-4171-b0c6-caebf726107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:39.183562 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984400af-56e9-4171-b0c6-caebf726107c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:39.183851 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984400af-56e9-4171-b0c6-caebf726107c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:39.184135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984400af-56e9-4171-b0c6-caebf726107c admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:39.184877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984400af-56e9-4171-b0c6-caebf726107c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:39.187315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984400af-56e9-4171-b0c6-caebf726107c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:39.210045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1413/2841] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:13:39 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:13:39.217042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82195f6f-4d4c-44dc-987a-d6a03c84af33 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:39.229302 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82195f6f-4d4c-44dc-987a-d6a03c84af33 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:39.234624 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678ac20-cd2f-49c9-88db-6a1628aefbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:39.235330 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678ac20-cd2f-49c9-88db-6a1628aefbf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:39.235735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678ac20-cd2f-49c9-88db-6a1628aefbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:39.236090 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678ac20-cd2f-49c9-88db-6a1628aefbf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:39.237059 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678ac20-cd2f-49c9-88db-6a1628aefbf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:39.238689 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678ac20-cd2f-49c9-88db-6a1628aefbf4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:39.250728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1429/2842] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:13:39.255171 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984400af-56e9-4171-b0c6-caebf726107c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:39.266071 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984400af-56e9-4171-b0c6-caebf726107c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:39.271557 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569b863-f21f-4909-8d0c-540e07a248e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:39.272513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569b863-f21f-4909-8d0c-540e07a248e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:39.272798 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569b863-f21f-4909-8d0c-540e07a248e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:39.273079 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569b863-f21f-4909-8d0c-540e07a248e8 admin admin] PATH_INFO: `/v3/projects/1d1f22ab046944b9a58c65925a2a941f/users/056ccf9a324643c3bdc7127b7b19a3c7/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:39.290350 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569b863-f21f-4909-8d0c-540e07a248e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1f22ab046944b9a58c65925a2a941f, user_id=056ccf9a324643c3bdc7127b7b19a3c7, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:39.293704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569b863-f21f-4909-8d0c-540e07a248e8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:39.309848 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1414/2843] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:13:39 2022] PUT /identity/v3/projects/1d1f22ab046944b9a58c65925a2a941f/users/056ccf9a324643c3bdc7127b7b19a3c7/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:13:39.313803 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678ac20-cd2f-49c9-88db-6a1628aefbf4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:39.325731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678ac20-cd2f-49c9-88db-6a1628aefbf4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:39.386643 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5e009-9972-44cc-87fb-e0763b2fa992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:39.387331 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e009-9972-44cc-87fb-e0763b2fa992 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:39.387859 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e009-9972-44cc-87fb-e0763b2fa992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:39.388139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e009-9972-44cc-87fb-e0763b2fa992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:39.389022 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5e009-9972-44cc-87fb-e0763b2fa992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:39.391273 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5e009-9972-44cc-87fb-e0763b2fa992 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:39.405533 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1430/2844] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:13:39.410077 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569b863-f21f-4909-8d0c-540e07a248e8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:39.421531 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569b863-f21f-4909-8d0c-540e07a248e8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:39.426616 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:39.427533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:39.427709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:39.427987 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] PATH_INFO: `/v3/projects/1d1f22ab046944b9a58c65925a2a941f/users/056ccf9a324643c3bdc7127b7b19a3c7/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:39.430014 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1f22ab046944b9a58c65925a2a941f, user_id=056ccf9a324643c3bdc7127b7b19a3c7, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:39.433307 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:39.444785 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1415/2845] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:13:39 2022] PUT /identity/v3/projects/1d1f22ab046944b9a58c65925a2a941f/users/056ccf9a324643c3bdc7127b7b19a3c7/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:13:43.903518 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73316072-3855-4d9a-b8a4-95932e211cd1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:43.903978 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73316072-3855-4d9a-b8a4-95932e211cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:43.904262 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73316072-3855-4d9a-b8a4-95932e211cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:43.932221 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-73316072-3855-4d9a-b8a4-95932e211cd1 None None] MFA Rules not processed for user `056ccf9a324643c3bdc7127b7b19a3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:13:43.933515 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-73316072-3855-4d9a-b8a4-95932e211cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:44.145640 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1431/2846] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:13:44.162503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:13:44.164451 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:44.282208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0398-e639-4efc-b5b9-6a321dc9735e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:13:44.288497 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401f972-2ecd-4eb9-a1cb-60109a905f53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:13:44.289162 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401f972-2ecd-4eb9-a1cb-60109a905f53 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:44.289944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401f972-2ecd-4eb9-a1cb-60109a905f53 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:44.289944 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401f972-2ecd-4eb9-a1cb-60109a905f53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:44.351129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401f972-2ecd-4eb9-a1cb-60109a905f53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:13:44.352861 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401f972-2ecd-4eb9-a1cb-60109a905f53 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:13:44.362513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1416/2847] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:13:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:13:46.353555 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b225e-b83e-4faa-a70e-1ed06f20fa16 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:13:46.355492 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b225e-b83e-4faa-a70e-1ed06f20fa16 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:13:46.355960 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b225e-b83e-4faa-a70e-1ed06f20fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:13:46.375805 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-607b225e-b83e-4faa-a70e-1ed06f20fa16 None None] MFA Rules not processed for user `7fce4b25aa88412590502ccbc8ce0690`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:13:46.377146 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-607b225e-b83e-4faa-a70e-1ed06f20fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:13:46.385818 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1432/2848] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:13:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:13:54.357938 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:13:56.368477 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:14:02.918470 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401f972-2ecd-4eb9-a1cb-60109a905f53 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:14:02.972896 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401f972-2ecd-4eb9-a1cb-60109a905f53 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:14:02.976500 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:14:02.977149 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:14:02.977435 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:14:02.977715 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:14:02.979510 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:14:02.989494 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:14:02.991057 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:14:02.999921 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1417/2849] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:14:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:14:10.308764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b225e-b83e-4faa-a70e-1ed06f20fa16 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:14:10.318712 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b225e-b83e-4faa-a70e-1ed06f20fa16 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:14:10.322277 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:14:10.322917 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:14:10.323205 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:14:10.323484 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:14:10.325952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:14:10.339322 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:14:10.341464 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:14:10.352421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1433/2850] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:14:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:14:12.995655 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=26 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:14:13.884218 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:14:13.893035 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bcac8-11a5-41f5-a3a7-c94e43d170c1 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:14:13.897366 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:14:13.898020 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:14:13.898306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:14:13.898588 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:14:13.900400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:14:13.914005 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:14:13.915456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:14:13.927151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1418/2851] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:14:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:14:20.076382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:14:20.085247 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf73239-3e4e-41c8-ab6a-e4391df159c0 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:14:20.088872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d124ca6-069a-4232-900d-82fd9bef5dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:14:20.089507 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d124ca6-069a-4232-900d-82fd9bef5dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:14:20.089788 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d124ca6-069a-4232-900d-82fd9bef5dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:14:20.090343 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d124ca6-069a-4232-900d-82fd9bef5dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:14:20.099499 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d124ca6-069a-4232-900d-82fd9bef5dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:14:20.100954 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d124ca6-069a-4232-900d-82fd9bef5dd3 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:14:20.111269 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1434/2852] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:14:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:14:23.922038 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:14:30.106122 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=12 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:15:11.667841 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:15:11.670013 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:15:11.681012 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb48a7-a93f-40d1-8e25-4d1d4e0ec924 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:15:11.684689 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:15:11.686059 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:15:11.686392 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:15:11.686670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:15:11.688443 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:15:11.698235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:15:11.699696 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:15:11.708209 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1419/2853] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:15:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:15:21.704323 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:16:34.817860 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d124ca6-069a-4232-900d-82fd9bef5dd3 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:16:34.855973 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d124ca6-069a-4232-900d-82fd9bef5dd3 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:16:34.859731 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:16:34.860413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:16:34.860699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:16:34.860977 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:16:34.862791 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:16:34.896402 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:16:34.898326 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:16:34.906867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1435/2854] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:16:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:16:34.983289 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:16:35.022100 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aed1be-bbcb-4d85-8c85-18768ccf4793 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:16:35.026666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:16:35.027697 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:16:35.028315 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:16:35.028601 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:16:35.030680 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:16:35.066660 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:16:35.068033 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:16:35.079521 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1420/2855] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:16:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:16:35.140222 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:16:35.141619 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:16:35.211099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece64b1c-7db6-4c31-9d04-1e8b10281711 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:16:35.216133 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:16:35.216807 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:16:35.217094 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:16:35.217372 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:16:35.219152 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:16:35.231153 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:16:35.232782 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:16:35.242660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1436/2856] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:16:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:16:36.875965 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:16:36.888894 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e96d5-86c7-4398-a53d-7c62999417c0 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:16:36.902327 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:16:36.903007 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:16:36.903300 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:16:36.903581 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:16:36.905592 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:16:36.919670 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:16:36.921302 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:16:36.935917 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1421/2857] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:16:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:16:45.238336 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=59 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:16:46.929393 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=33 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:01.655387 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:01.657163 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:01.708109 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3cc971-4db1-43d3-9933-32398714990d service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:01.714000 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:01.714862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:01.715250 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:01.715608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:01.718180 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:01.732013 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:01.733369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:01.746424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1437/2858] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:18:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:18:04.159650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:04.190845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2233d92-f061-4e55-8809-2b9ccc2f4769 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:04.197045 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:04.197706 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:04.197995 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:04.198277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:04.200086 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:04.214208 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:04.215666 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:04.233356 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1422/2859] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:18:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:18:08.614176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:08.637301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c56d08-6146-4808-9441-3fd75789466b service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:08.649704 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:08.650577 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:08.650715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:08.650972 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:08.656519 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:08.711862 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:08.713557 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:08.724699 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1438/2860] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:18:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:18:11.823128 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:11.836384 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7bd9f-a97d-410f-ac98-3df5e2c9dbc3 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:11.842883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:11.843085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:11.843400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:11.843676 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:11.845558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:11.863973 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:11.865625 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:11.881599 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1423/2861] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:18:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:18:11.886376 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:11.974066 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c38055-56e8-4ee0-be3e-393196ecbfd5 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:11.978588 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:11.979453 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:11.979728 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:11.980004 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] PATH_INFO: `/v3/users/7fce4b25aa88412590502ccbc8ce0690` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:11.981217 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fce4b25aa88412590502ccbc8ce0690)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:11.983378 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:12.028669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fce4b25aa88412590502ccbc8ce0690'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.040362 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fce4b25aa88412590502ccbc8ce0690'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.041034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fce4b25aa88412590502ccbc8ce0690'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.041637 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fce4b25aa88412590502ccbc8ce0690'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.046872 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:12.048995 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fce4b25aa88412590502ccbc8ce0690'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.062887 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1439/2862] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:18:11 2022] DELETE /identity/v3/users/7fce4b25aa88412590502ccbc8ce0690 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:18:12.239723 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:12.254188 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf50c7b-c868-47a7-8371-b5ca202fdd9b service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:12.317943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:12.318818 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:12.319103 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:12.319490 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] PATH_INFO: `/v3/projects/7ab20a59f54348e6b0d892ea010aff9c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:12.320739 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab20a59f54348e6b0d892ea010aff9c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:12.322496 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:12.376801 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab20a59f54348e6b0d892ea010aff9c', 'request_id': 'req-530490ae-8d1f-44a9-b271-cf9efe979358'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.383415 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab20a59f54348e6b0d892ea010aff9c', 'request_id': 'req-530490ae-8d1f-44a9-b271-cf9efe979358'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.390140 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab20a59f54348e6b0d892ea010aff9c', 'request_id': 'req-530490ae-8d1f-44a9-b271-cf9efe979358'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.418056 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] The token cache is being invalidate because project 7ab20a59f54348e6b0d892ea010aff9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:18:12.418501 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.419777 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.421627 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1424/2863] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:18:12 2022] DELETE /identity/v3/projects/7ab20a59f54348e6b0d892ea010aff9c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:18:12.425773 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:12.427216 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:12.441099 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf39a97-58cb-47c7-b284-15382604817d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:12.502670 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:12.503548 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:12.503832 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:12.504106 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] PATH_INFO: `/v3/users/056ccf9a324643c3bdc7127b7b19a3c7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:12.505371 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=056ccf9a324643c3bdc7127b7b19a3c7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:12.507306 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:12.557968 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '056ccf9a324643c3bdc7127b7b19a3c7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.571997 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '056ccf9a324643c3bdc7127b7b19a3c7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.572764 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '056ccf9a324643c3bdc7127b7b19a3c7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.574019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '056ccf9a324643c3bdc7127b7b19a3c7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.582219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-281500f0-3248-4a9e-a141-6866853def99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '056ccf9a324643c3bdc7127b7b19a3c7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.596653 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1440/2864] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:18:12 2022] DELETE /identity/v3/users/056ccf9a324643c3bdc7127b7b19a3c7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:18:12.772879 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:12.774600 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:12.788579 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530490ae-8d1f-44a9-b271-cf9efe979358 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:12.823157 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:12.848907 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:12.849884 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:12.850174 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:12.850464 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] PATH_INFO: `/v3/projects/1d1f22ab046944b9a58c65925a2a941f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:12.851834 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1f22ab046944b9a58c65925a2a941f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:12.853786 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:12.902528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1f22ab046944b9a58c65925a2a941f', 'request_id': 'req-0ec95b79-b624-4538-a1e3-2688ce680ab8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.909788 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1f22ab046944b9a58c65925a2a941f', 'request_id': 'req-0ec95b79-b624-4538-a1e3-2688ce680ab8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.915372 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1f22ab046944b9a58c65925a2a941f', 'request_id': 'req-0ec95b79-b624-4538-a1e3-2688ce680ab8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.928961 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] The token cache is being invalidate because project 1d1f22ab046944b9a58c65925a2a941f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:18:12.929253 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.929867 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0ec95b79-b624-4538-a1e3-2688ce680ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:18:12.931582 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1425/2865] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:18:12 2022] DELETE /identity/v3/projects/1d1f22ab046944b9a58c65925a2a941f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:18:12.936579 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3412e-747e-4cb4-93a4-1f9d4387b387 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:12.937073 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3412e-747e-4cb4-93a4-1f9d4387b387 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:12.937176 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3412e-747e-4cb4-93a4-1f9d4387b387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:12.993320 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-10e3412e-747e-4cb4-93a4-1f9d4387b387 None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:18:12.994511 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-10e3412e-747e-4cb4-93a4-1f9d4387b387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:13.026919 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:13.056622 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1441/2866] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:18:13.064373 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b246cf-fc69-4a45-b5aa-65c6e20ee2bb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.064667 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b246cf-fc69-4a45-b5aa-65c6e20ee2bb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.064953 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b246cf-fc69-4a45-b5aa-65c6e20ee2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.083754 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-73b246cf-fc69-4a45-b5aa-65c6e20ee2bb None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:18:13.084998 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-73b246cf-fc69-4a45-b5aa-65c6e20ee2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:13.093704 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1426/2867] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:18:13.097893 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3412e-747e-4cb4-93a4-1f9d4387b387 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:13.107669 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3412e-747e-4cb4-93a4-1f9d4387b387 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:13.113567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:13.114239 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.114524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.114809 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.115544 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 30 12:18:13.115856 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:13.118789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:13.132189 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1442/2868] 10.222.0.32 () {58 vars in 1152 bytes} [Mon May 30 12:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 30 12:18:13.136586 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b246cf-fc69-4a45-b5aa-65c6e20ee2bb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:13.146797 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b246cf-fc69-4a45-b5aa-65c6e20ee2bb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:13.151694 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751027b-29f2-4b3d-82bd-77029c0d30af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:13.152370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751027b-29f2-4b3d-82bd-77029c0d30af admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.152707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751027b-29f2-4b3d-82bd-77029c0d30af admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.152929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751027b-29f2-4b3d-82bd-77029c0d30af admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.154230 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751027b-29f2-4b3d-82bd-77029c0d30af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:13.156534 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751027b-29f2-4b3d-82bd-77029c0d30af admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:13.185181 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1427/2869] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:18:13 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:18:13.186313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:13.196811 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a91f3-8b5b-47da-bb0f-c7e149b71009 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:13.254681 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83e78-6287-4022-894c-b3ca6d14e5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:13.255363 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83e78-6287-4022-894c-b3ca6d14e5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.255651 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83e78-6287-4022-894c-b3ca6d14e5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.256042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83e78-6287-4022-894c-b3ca6d14e5fb admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.256680 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e83e78-6287-4022-894c-b3ca6d14e5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:13.258608 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e83e78-6287-4022-894c-b3ca6d14e5fb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:13.280844 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1443/2870] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:18:13 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:18:13.284916 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751027b-29f2-4b3d-82bd-77029c0d30af admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:13.295414 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751027b-29f2-4b3d-82bd-77029c0d30af admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:13.300605 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac23c10-74b3-44fd-a937-e37a9473560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:13.301277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23c10-74b3-44fd-a937-e37a9473560c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.301558 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23c10-74b3-44fd-a937-e37a9473560c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.301828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23c10-74b3-44fd-a937-e37a9473560c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.302631 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac23c10-74b3-44fd-a937-e37a9473560c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:13.304134 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac23c10-74b3-44fd-a937-e37a9473560c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:13.315002 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1428/2871] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:18:13.319657 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83e78-6287-4022-894c-b3ca6d14e5fb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:13.330129 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83e78-6287-4022-894c-b3ca6d14e5fb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:13.334649 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23a465-7ef7-4e1c-b8b6-40d374b0d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:13.335301 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23a465-7ef7-4e1c-b8b6-40d374b0d05d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.335584 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23a465-7ef7-4e1c-b8b6-40d374b0d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.335867 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23a465-7ef7-4e1c-b8b6-40d374b0d05d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.336625 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23a465-7ef7-4e1c-b8b6-40d374b0d05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:13.338219 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23a465-7ef7-4e1c-b8b6-40d374b0d05d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:13.348987 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1444/2872] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:18:13.353238 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac23c10-74b3-44fd-a937-e37a9473560c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:13.363693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac23c10-74b3-44fd-a937-e37a9473560c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:13.369031 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:13.369925 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.370213 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.370498 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] PATH_INFO: `/v3/projects/5143c400e4544c79955f901394c449f6/users/923af8dc0f1d4779bf388151093311f6/roles/932e429fa28248f285663a90fc880876` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.386456 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5143c400e4544c79955f901394c449f6, user_id=923af8dc0f1d4779bf388151093311f6, role_id=932e429fa28248f285663a90fc880876)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:13.389755 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:13.405533 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1429/2873] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:18:13 2022] PUT /identity/v3/projects/5143c400e4544c79955f901394c449f6/users/923af8dc0f1d4779bf388151093311f6/roles/932e429fa28248f285663a90fc880876 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:18:13.410522 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c41f7b-01f4-4fff-b3e7-f9808542a35c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.410840 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c41f7b-01f4-4fff-b3e7-f9808542a35c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.411154 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c41f7b-01f4-4fff-b3e7-f9808542a35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.429102 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-73c41f7b-01f4-4fff-b3e7-f9808542a35c None None] MFA Rules not processed for user `a8ba729f356d47049a450132ac3851da`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:18:13.430248 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-73c41f7b-01f4-4fff-b3e7-f9808542a35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:13.492562 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1445/2874] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:18:13.502876 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:13.504640 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:13.563132 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa0f93-4499-4111-8f7a-2c0697dc0107 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:13.566837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce86aa0-3ee6-4e0b-a417-907e00dbedc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:13.567482 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce86aa0-3ee6-4e0b-a417-907e00dbedc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:13.567769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce86aa0-3ee6-4e0b-a417-907e00dbedc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:13.568131 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce86aa0-3ee6-4e0b-a417-907e00dbedc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:13.578001 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce86aa0-3ee6-4e0b-a417-907e00dbedc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:13.579440 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce86aa0-3ee6-4e0b-a417-907e00dbedc9 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:13.589276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1430/2875] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:18:17.650636 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede44c4-54f4-46e1-bbe5-9e98249dd5bc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:17.651236 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede44c4-54f4-46e1-bbe5-9e98249dd5bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:17.651354 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede44c4-54f4-46e1-bbe5-9e98249dd5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:17.679355 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4ede44c4-54f4-46e1-bbe5-9e98249dd5bc None None] MFA Rules not processed for user `923af8dc0f1d4779bf388151093311f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:18:17.680671 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4ede44c4-54f4-46e1-bbe5-9e98249dd5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:17.715148 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:17.750810 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1446/2876] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:18:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:18:17.755444 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce86aa0-3ee6-4e0b-a417-907e00dbedc9 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:17.779143 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce86aa0-3ee6-4e0b-a417-907e00dbedc9 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:17.789003 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad342999-fe16-46f2-aed4-9555ac20a296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:17.789693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad342999-fe16-46f2-aed4-9555ac20a296 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:17.790029 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad342999-fe16-46f2-aed4-9555ac20a296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:17.790255 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad342999-fe16-46f2-aed4-9555ac20a296 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:17.790966 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad342999-fe16-46f2-aed4-9555ac20a296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:17.792932 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad342999-fe16-46f2-aed4-9555ac20a296 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:17.796528 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:17.827845 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1431/2877] 10.222.0.32 () {60 vars in 1152 bytes} [Mon May 30 12:18:17 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 30 12:18:17.831947 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede44c4-54f4-46e1-bbe5-9e98249dd5bc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:17.843307 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede44c4-54f4-46e1-bbe5-9e98249dd5bc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:17.909537 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:17.910350 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:17.910710 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:17.911001 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:17.911723 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:17.913711 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:17.942804 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1447/2878] 10.222.0.32 () {60 vars in 1143 bytes} [Mon May 30 12:18:17 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 30 12:18:17.946977 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad342999-fe16-46f2-aed4-9555ac20a296 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:17.958765 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad342999-fe16-46f2-aed4-9555ac20a296 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:17.964931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeaf43-cfe2-47e8-977e-9c86a7741f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:17.965634 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeaf43-cfe2-47e8-977e-9c86a7741f0d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:17.965942 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeaf43-cfe2-47e8-977e-9c86a7741f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:17.966235 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeaf43-cfe2-47e8-977e-9c86a7741f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:17.966985 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeaf43-cfe2-47e8-977e-9c86a7741f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:17.968693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeaf43-cfe2-47e8-977e-9c86a7741f0d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:17.972037 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:17.973814 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:17.980770 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1432/2879] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:18:17.985360 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeaf43-cfe2-47e8-977e-9c86a7741f0d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:17.996608 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeaf43-cfe2-47e8-977e-9c86a7741f0d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:18.002577 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5c331-3a1d-4862-8294-24d99515c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:18.003491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5c331-3a1d-4862-8294-24d99515c692 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:18.003779 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5c331-3a1d-4862-8294-24d99515c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:18.004065 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5c331-3a1d-4862-8294-24d99515c692 admin admin] PATH_INFO: `/v3/projects/c2db5b2875554c89990a99480a911a02/users/eff2e5db745f4d66881d3a65e7b1a4a5/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:18.022611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5c331-3a1d-4862-8294-24d99515c692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2db5b2875554c89990a99480a911a02, user_id=eff2e5db745f4d66881d3a65e7b1a4a5, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:18.025865 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5c331-3a1d-4862-8294-24d99515c692 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:18.041513 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1433/2880] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:18:17 2022] PUT /identity/v3/projects/c2db5b2875554c89990a99480a911a02/users/eff2e5db745f4d66881d3a65e7b1a4a5/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:18:18.046112 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5c331-3a1d-4862-8294-24d99515c692 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:18.061975 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa650f3-f7e9-439f-be35-beb22d56ce92 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:18.124569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5c331-3a1d-4862-8294-24d99515c692 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:18.129623 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d3b0e-0223-405e-b845-755613606207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:18.130306 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d3b0e-0223-405e-b845-755613606207 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:18.130598 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d3b0e-0223-405e-b845-755613606207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:18.130875 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d3b0e-0223-405e-b845-755613606207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:18.131678 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d3b0e-0223-405e-b845-755613606207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:18.133254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d3b0e-0223-405e-b845-755613606207 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:18.137364 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246be9bf-6cd9-41eb-b9e3-a9d53c3c9403 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:18.138048 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246be9bf-6cd9-41eb-b9e3-a9d53c3c9403 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:18.138454 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246be9bf-6cd9-41eb-b9e3-a9d53c3c9403 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:18.138952 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246be9bf-6cd9-41eb-b9e3-a9d53c3c9403 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:18.141369 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-246be9bf-6cd9-41eb-b9e3-a9d53c3c9403 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:18.144437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1434/2881] 10.222.0.32 () {58 vars in 1121 bytes} [Mon May 30 12:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 30 12:18:18.148348 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d3b0e-0223-405e-b845-755613606207 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:18.153032 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:18.157282 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246be9bf-6cd9-41eb-b9e3-a9d53c3c9403 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:18.159411 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246be9bf-6cd9-41eb-b9e3-a9d53c3c9403 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:18.161828 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d3b0e-0223-405e-b845-755613606207 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:18.167018 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:18.167929 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:18.168379 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:18.168537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] PATH_INFO: `/v3/projects/c2db5b2875554c89990a99480a911a02/users/eff2e5db745f4d66881d3a65e7b1a4a5/roles/4d21498572b14478a996ac538be5382e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:18.171342 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2db5b2875554c89990a99480a911a02, user_id=eff2e5db745f4d66881d3a65e7b1a4a5, role_id=4d21498572b14478a996ac538be5382e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:18.173707 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1448/2882] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:18:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:18:18.174118 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:18.186650 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1435/2883] 10.222.0.32 () {60 vars in 1482 bytes} [Mon May 30 12:18:18 2022] PUT /identity/v3/projects/c2db5b2875554c89990a99480a911a02/users/eff2e5db745f4d66881d3a65e7b1a4a5/roles/4d21498572b14478a996ac538be5382e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 30 12:18:22.782169 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795cba5-9a49-4555-a48b-c88cbf2a2d72 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:22.782765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795cba5-9a49-4555-a48b-c88cbf2a2d72 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:22.782765 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795cba5-9a49-4555-a48b-c88cbf2a2d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:22.812719 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d795cba5-9a49-4555-a48b-c88cbf2a2d72 None None] MFA Rules not processed for user `eff2e5db745f4d66881d3a65e7b1a4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:18:22.813950 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d795cba5-9a49-4555-a48b-c88cbf2a2d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:22.887986 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1449/2884] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:18:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:18:22.904931 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:18:22.906935 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:22.991486 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40716e2c-aae8-4d17-9253-7227165ac39e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:18:22.998168 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d2456-6749-4ad4-b985-53802f24bab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:18:22.998837 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2456-6749-4ad4-b985-53802f24bab3 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:22.999129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2456-6749-4ad4-b985-53802f24bab3 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:22.999406 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d2456-6749-4ad4-b985-53802f24bab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:23.033757 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:23.064693 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d2456-6749-4ad4-b985-53802f24bab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:18:23.067286 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d2456-6749-4ad4-b985-53802f24bab3 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:18:23.077116 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1436/2885] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:18:32.884735 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=36 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:33.072546 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=14 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:18:41.514225 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad939718-5fd9-45b6-9fab-4998cf78a307 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:18:41.514774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad939718-5fd9-45b6-9fab-4998cf78a307 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:18:41.514774 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad939718-5fd9-45b6-9fab-4998cf78a307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:18:41.532935 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ad939718-5fd9-45b6-9fab-4998cf78a307 None None] MFA Rules not processed for user `923af8dc0f1d4779bf388151093311f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:18:41.534137 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ad939718-5fd9-45b6-9fab-4998cf78a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:18:41.541034 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1450/2886] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:18:51.527160 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:19:02.098123 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d2456-6749-4ad4-b985-53802f24bab3 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:19:02.153139 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d2456-6749-4ad4-b985-53802f24bab3 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:19:02.156775 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c534f15-1c86-469e-b771-e561566630c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:19:02.157424 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c534f15-1c86-469e-b771-e561566630c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:19:02.157709 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c534f15-1c86-469e-b771-e561566630c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:19:02.157991 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c534f15-1c86-469e-b771-e561566630c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:19:02.167151 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c534f15-1c86-469e-b771-e561566630c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:19:02.168551 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c534f15-1c86-469e-b771-e561566630c3 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:19:02.178504 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1437/2887] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:19:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:19:12.174254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=26 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:20:14.681019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad939718-5fd9-45b6-9fab-4998cf78a307 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:20:14.682443 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ad939718-5fd9-45b6-9fab-4998cf78a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:20:14.694980 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad939718-5fd9-45b6-9fab-4998cf78a307 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:20:14.698592 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:20:14.699230 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:20:14.699513 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:20:14.699789 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:20:14.701560 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:20:14.711943 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:20:14.713397 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:20:14.721252 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1451/2888] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:20:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:20:24.717780 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:21:02.877548 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c534f15-1c86-469e-b771-e561566630c3 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:21:02.887068 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c534f15-1c86-469e-b771-e561566630c3 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:21:02.890889 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d34bf-b18c-4170-ba5a-16921de85ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:21:02.891537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d34bf-b18c-4170-ba5a-16921de85ceb service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:21:02.891823 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d34bf-b18c-4170-ba5a-16921de85ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:21:02.892105 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d34bf-b18c-4170-ba5a-16921de85ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:21:02.893928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7d0d34bf-b18c-4170-ba5a-16921de85ceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:21:02.907816 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d34bf-b18c-4170-ba5a-16921de85ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:21:02.909276 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d34bf-b18c-4170-ba5a-16921de85ceb service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:21:02.920928 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1438/2889] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:21:04.098423 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:21:04.107585 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41de020-7163-447b-aa2c-1364c2a4527a service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:21:04.112019 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:21:04.112696 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:21:04.113016 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:21:04.113335 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:21:04.116530 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:21:04.129886 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:21:04.131985 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:21:04.143535 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1452/2890] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:21:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 30 12:21:06.421461 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26509-7632-49b6-98eb-67b29abbab15 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:21:06.421971 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26509-7632-49b6-98eb-67b29abbab15 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:21:06.422195 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e26509-7632-49b6-98eb-67b29abbab15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:21:06.441290 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-99e26509-7632-49b6-98eb-67b29abbab15 None None] MFA Rules not processed for user `923af8dc0f1d4779bf388151093311f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:21:06.442611 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-99e26509-7632-49b6-98eb-67b29abbab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:21:06.450607 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1439/2891] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:21:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:21:07.048726 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:21:07.058485 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9027649-a591-4196-85c4-4e10e38fa7ec service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:21:07.062179 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:21:07.062824 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:21:07.063111 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:21:07.063401 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:21:07.065223 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:21:07.075496 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:21:07.076969 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:21:07.086033 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1453/2892] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:21:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:21:16.436127 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:21:17.081538 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=12 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:23:02.184503 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382211a-8c54-444e-897b-b3823fab975d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:23:02.185552 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382211a-8c54-444e-897b-b3823fab975d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:23:02.185868 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382211a-8c54-444e-897b-b3823fab975d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:23:02.204229 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9382211a-8c54-444e-897b-b3823fab975d None None] MFA Rules not processed for user `923af8dc0f1d4779bf388151093311f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 30 12:23:02.205446 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9382211a-8c54-444e-897b-b3823fab975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:23:02.212303 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1440/2893] 10.222.0.32 () {58 vars in 957 bytes} [Mon May 30 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 30 12:23:02.981424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:23:03.051049 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f5903-d7bb-467d-91e3-ba9e0aa4de09 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:23:03.056184 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:23:03.056850 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:23:03.057139 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:23:03.057421 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:23:03.059238 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:23:03.071424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:23:03.073050 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:23:03.083549 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1454/2894] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:23:05.464334 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382211a-8c54-444e-897b-b3823fab975d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:23:05.478812 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382211a-8c54-444e-897b-b3823fab975d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:23:05.489268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:23:05.490254 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:23:05.490707 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:23:05.491353 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:23:05.494270 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:23:05.514575 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:23:05.516450 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:23:05.531363 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1441/2895] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:23:13.079242 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=32 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:23:15.525161 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:23:16.824409 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:23:16.838849 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f579c-cffd-4d79-8f94-8b57e0a6c72f service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:23:16.844413 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:23:16.845065 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:23:16.845512 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:23:16.845839 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:23:16.848604 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:23:16.862589 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:23:16.864234 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:23:16.875914 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1455/2896] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:23:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:23:20.932085 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:23:20.943859 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f3ebe-af76-486b-b738-447083792e6e service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:23:20.948969 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614722f-7237-4b23-b3d7-09895c748f53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:23:20.949787 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614722f-7237-4b23-b3d7-09895c748f53 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:23:20.950064 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614722f-7237-4b23-b3d7-09895c748f53 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:23:20.950346 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614722f-7237-4b23-b3d7-09895c748f53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:23:20.962108 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614722f-7237-4b23-b3d7-09895c748f53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:23:20.964247 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614722f-7237-4b23-b3d7-09895c748f53 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:23:20.976377 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1442/2897] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:23:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:23:21.470752 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:23:21.472313 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:23:21.486162 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04501a55-39aa-478d-91b6-1772471f2f6a service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:23:21.492420 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70dce2f0284847ab8f07f55d700d0f60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:23:21.493084 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:23:21.493405 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:23:21.493660 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:23:21.495848 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:23:21.509524 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:23:21.511746 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:23:21.523294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1456/2898] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:23:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:23:30.970983 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:23:31.518456 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=12 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:25:00.841753 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614722f-7237-4b23-b3d7-09895c748f53 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:25:00.851310 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614722f-7237-4b23-b3d7-09895c748f53 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:25:00.854922 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4ce98-6429-45f3-a719-59b2f5533a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:25:00.855563 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4ce98-6429-45f3-a719-59b2f5533a27 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:25:00.855849 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4ce98-6429-45f3-a719-59b2f5533a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:25:00.856129 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4ce98-6429-45f3-a719-59b2f5533a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:25:00.864671 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4ce98-6429-45f3-a719-59b2f5533a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:25:00.866400 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4ce98-6429-45f3-a719-59b2f5533a27 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:25:00.875491 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1443/2899] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:25:10.871172 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:25:13.938382 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:25:13.948117 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51893e2e-115b-4a4c-8f12-50c53f651b21 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:25:13.951883 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2d95e-5480-4097-a93d-642f8ee401b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ea38389eee4472a8917023643be9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:25:13.952574 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2d95e-5480-4097-a93d-642f8ee401b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:25:13.952863 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2d95e-5480-4097-a93d-642f8ee401b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:25:13.953145 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2d95e-5480-4097-a93d-642f8ee401b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:25:14.016966 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2d95e-5480-4097-a93d-642f8ee401b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:25:14.018715 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2d95e-5480-4097-a93d-642f8ee401b1 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:25:14.030045 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1457/2900] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:25:17.173058 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4ce98-6429-45f3-a719-59b2f5533a27 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:25:17.186629 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4ce98-6429-45f3-a719-59b2f5533a27 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:25:17.196370 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:25:17.197032 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:25:17.197949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:25:17.197949 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:25:17.199883 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:25:17.215277 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:25:17.217268 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:25:17.232652 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1444/2901] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:25:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:25:20.432621 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2d95e-5480-4097-a93d-642f8ee401b1 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:25:20.449854 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2d95e-5480-4097-a93d-642f8ee401b1 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:25:20.454655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:25:20.455536 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:25:20.455822 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:25:20.456098 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] PATH_INFO: `/v3/users/923af8dc0f1d4779bf388151093311f6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:25:20.457304 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=923af8dc0f1d4779bf388151093311f6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:25:20.459200 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:25:20.506843 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '923af8dc0f1d4779bf388151093311f6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.521091 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '923af8dc0f1d4779bf388151093311f6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.521816 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '923af8dc0f1d4779bf388151093311f6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.522424 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '923af8dc0f1d4779bf388151093311f6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.529802 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '923af8dc0f1d4779bf388151093311f6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.542647 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1458/2902] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:25:20 2022] DELETE /identity/v3/users/923af8dc0f1d4779bf388151093311f6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:25:20.740267 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:25:20.752299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb299fe4-8024-40d1-8c8d-a5cefa943707 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:25:20.810283 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:25:20.811154 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:25:20.811437 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:25:20.811719 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] PATH_INFO: `/v3/projects/5143c400e4544c79955f901394c449f6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:25:20.812937 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5143c400e4544c79955f901394c449f6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:25:20.814847 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:25:20.861293 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5143c400e4544c79955f901394c449f6', 'request_id': 'req-81522b61-3f68-4aef-a353-0605cd3fcc0b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.868135 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5143c400e4544c79955f901394c449f6', 'request_id': 'req-81522b61-3f68-4aef-a353-0605cd3fcc0b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.873890 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5143c400e4544c79955f901394c449f6', 'request_id': 'req-81522b61-3f68-4aef-a353-0605cd3fcc0b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.891537 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] The token cache is being invalidate because project 5143c400e4544c79955f901394c449f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:25:20.891833 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.892487 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:20.894104 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1445/2903] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:25:20 2022] DELETE /identity/v3/projects/5143c400e4544c79955f901394c449f6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:25:20.898447 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:25:20.900112 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:25:20.913903 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6814d-f04e-43f4-93e1-eb6f91601303 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:25:20.982892 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:25:20.983787 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:25:20.984068 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:25:20.984373 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] PATH_INFO: `/v3/users/eff2e5db745f4d66881d3a65e7b1a4a5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:25:20.985687 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff2e5db745f4d66881d3a65e7b1a4a5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:25:20.987679 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:25:20.994837 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:25:21.040851 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff2e5db745f4d66881d3a65e7b1a4a5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.052567 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff2e5db745f4d66881d3a65e7b1a4a5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.053266 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff2e5db745f4d66881d3a65e7b1a4a5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.054601 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff2e5db745f4d66881d3a65e7b1a4a5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.061700 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff2e5db745f4d66881d3a65e7b1a4a5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.076271 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1459/2904] 10.222.0.32 () {58 vars in 1223 bytes} [Mon May 30 12:25:20 2022] DELETE /identity/v3/users/eff2e5db745f4d66881d3a65e7b1a4a5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:25:21.250166 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:25:21.251769 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:25:21.267569 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81522b61-3f68-4aef-a353-0605cd3fcc0b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:25:21.332877 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8ba729f356d47049a450132ac3851da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '41eaf29f1f484186a95e357823daad72', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:25:21.333740 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:25:21.334026 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:25:21.334299 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] PATH_INFO: `/v3/projects/c2db5b2875554c89990a99480a911a02` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:25:21.336006 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2db5b2875554c89990a99480a911a02)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:25:21.337978 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:25:21.359317 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:25:21.386239 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2db5b2875554c89990a99480a911a02', 'request_id': 'req-3306d120-0d9b-4eec-af04-e010a60b8b7c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.392702 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2db5b2875554c89990a99480a911a02', 'request_id': 'req-3306d120-0d9b-4eec-af04-e010a60b8b7c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.398948 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2db5b2875554c89990a99480a911a02', 'request_id': 'req-3306d120-0d9b-4eec-af04-e010a60b8b7c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.411954 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] The token cache is being invalidate because project c2db5b2875554c89990a99480a911a02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 30 12:25:21.412311 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.412943 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3306d120-0d9b-4eec-af04-e010a60b8b7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 30 12:25:21.415200 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1446/2905] 10.222.0.32 () {58 vars in 1232 bytes} [Mon May 30 12:25:21 2022] DELETE /identity/v3/projects/c2db5b2875554c89990a99480a911a02 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 30 12:25:31.071963 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:25:31.409004 dv-cinder-840512-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 30 12:28:22.956232 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 30 12:28:22.957655 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:28:23.097333 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19bb70-4ec8-4e37-8253-8ee606d950ae admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 30 12:28:23.157384 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7252d-1e55-47f0-bfe7-b6bffe879969 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c654dad1f844fe9995c594116619216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ccd54f1160f441ff95c6a7a1efc923df', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 30 12:28:23.158042 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7252d-1e55-47f0-bfe7-b6bffe879969 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 30 12:28:23.158334 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7252d-1e55-47f0-bfe7-b6bffe879969 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 30 12:28:23.158615 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7252d-1e55-47f0-bfe7-b6bffe879969 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 30 12:28:23.160465 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cca7252d-1e55-47f0-bfe7-b6bffe879969 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 30 12:28:23.171672 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7252d-1e55-47f0-bfe7-b6bffe879969 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 30 12:28:23.173294 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7252d-1e55-47f0-bfe7-b6bffe879969 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 30 12:28:23.183171 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1460/2906] 10.222.0.32 () {60 vars in 1365 bytes} [Mon May 30 12:28:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 30 12:28:33.178956 dv-cinder-840512-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}